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Abstract 


In the Fall 2010 semester, student enrollment in distance education courses increased in the 
United States to over 6.1 million students taking at least one distance course. Distance education 
allows institutions to meet increasing demands from the government and business sectors for 
more graduates in ways that face-to-face courses cannot meet with physical space, faculty 
resources, and class size limitations. Faculty and administrators express concerns about poor 
academic quality in online courses and programs. Existing models such as Sloan-C Five Pillars 
of Quality Online Education and Quality Matters do not adequately address course delivery 
quality. This Formative Research study was an effort to develop an instructional delivery 
framework for online courses that would meet requirements from (a) accreditation and regulatory 
agencies, (b) faculty and administration concerns about how to design and implement quality 
distance courses, and (c) maintenance of pedagogical flexibility for faculty. The study produced 
a quality instructional delivery model that was developed through iterative cycles of faculty use 
and feedback with the framework. Data were analyzed using the Three C’s Model. This 
developed model provides a flexible, comprehensive, and quality-oriented model for the delivery 
of distance courses in a traditional university environment. The framework enables faculty of 
various disciplines and experience with online instruction to design a course for distance delivery 
that meets quality demands while allowing for the freedom customary for faculty membership. 
The framework produced an appealing, effective, and efficient instrument that can support the 
growth of quality distance courses and programs within institutions. 
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Chapter 1: Introduction 

Smith (2011) indicated that online education allows institutions to meet increasing 
demands from the government and business sectors for more graduates in ways that face-to-face 
courses cannot meet with physical space, faculty resource, and class size limitations. 

In the Fall 2010 semester, student enrollment in distance education courses increased in the 
United States (U.S.) by approximately 560,000 over the previous year to over 6.1 million 
students taking at least one distance course (Allen & Seaman, 201 1). This growth in online 
enrollment was slower than in the previous decade, but still exceeded traditional face-to-face 
enrollments in the same period (Allen & Seaman, 2011). The present study resulted in the 
development of an instructional delivery framework for distance education courses at a small 
private liberal arts university in western New York state that meets requirements from 
accreditation and regulatory agencies, faculty and administration concerns, and maintenance of 
pedagogical flexibility for faculty. 

The small liberal arts institution, herein referred to with the pseudonym Magdalene 
University or Magdalene, that is the target of the study has increased the number of distance 
education courses from 9 in the 2002-03 academic year to 55 in the 2009-10 academic year (P. 

L. Beaman, personal communication, April 10, 2012). Administrators are seeking to increase the 
number of distance education courses to 25% of all courses offered. Magdalene University 
faculty at are concerned that the expansion of distance education may not be associated with 
adequate support or attention to issues of quality and desire a system that, while maintaining 
academic quality and rigor, will support faculty development of distance education courses (M. 

S. Brogan, personal communication, July 22, 2011). The subject of the Formative Research 
study was the development of a quality course delivery framework upon which the expansion of 
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distance courses and programs could take place and that reflects the combined objectives of 
faculty. 

Background of the Problem 

A recent indicator of growth for distance education is the advent of Massively Open 
Online Courses or MOOCs (Vardi, 2012). In the Fall 2011 semester, approximately 450,000 
students enrolled in three computer science courses offered by Stanford University via the 
MOOC format (Vardi, 2012). Smith (2011) indicated that university leadership views booming 
enrollment in online programs as one reason to consider adding or expanding online programs at 
their respective institutions. Despite the growth in online enrollments across the higher 
education industry, distance education scholars submit that administrators and faculty continue to 
indicate concern with online course quality (Austin, 2010; Endean, Bin, & Ruo, 2010; Forsyth, 
Pizzica, Laxton, & Mahony, 2010; Hoskins, 2009; Kee Meng & Mayadas, 2010; Picciano, 2009; 
P. S. Smith, 2011; Westerfelt, 2011; Wickersham & McElhany, 2010). Harvey and Williams 
(2010) posited that higher education regulators and accreditors also reported concern with quality 
and increased oversight in this modality of higher education. Faculty and administrators show an 
interest in achieving and monitoring quality for online education in response to institutional 
planning, online enrollment growth, and accreditation, but existing models including regional 
accreditation guidelines, Sloan-C Five Pillars of Quality Distance Fearning, and Quality Matters 
do not address quality at the course delivery level (Battin-Fittle, 2009; Bourne, Harris, & 
Mayadas, 2005; Pollacia & McCallister, 2009; Westerfelt, 2011; Wickersham & McElhany, 
2010). 

Smith (2011) noted that within colleges and universities the responsibility for 
instructional quality traditionally has resided with faculty and departments, thereby creating 
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disparities and inconsistencies in institutional reporting of quality. Such practices also pertain to 
the delivery of distance education courses as an extension of the department offerings or 
programs (Smith, 2011). Driven by the increased scrutiny and oversight of distance programs by 
accrediting agencies for higher education, the need for quality assessment has moved from a 
faculty or departmental approach to a more institution-wide approach (P. S. Smith, 2011). 
Traditionally campus-focused institutions lack the ability to monitor effectively distance course 
and program quality, but concomitantly recognize online quality as one element of a school’s 
academic reputation (Forsyth et al., 2010). Traditional structures, resources, processes, and 
governance practices are not effectively equipped to meet the quickly evolving practices and 
regulation standards for evaluating online program quality (Endean et al., 2010; P. S. Smith, 
2011 ). 

Some distance education scholars proposed that to be effective and efficient quality 
evaluation should include systematic resources, support, and communication (Forsyth et al., 
2010). Technology-based quality monitoring tools and rubrics enhance the ability of schools to 
create and sufficiently evaluate course quality in distance learning programs (Postek, Ledziska, 

& Czarkowski, 2010; P. S. Smith, 2011). The lack of quality standards presents challenges with 
accreditation, compliance, and a quality student experience (Endean et al., 2010). Quality 
measurement instruments do not exist for institutions new to teaching or monitoring quality 
online courses (Endean et al., 2010). 

Westerfelt (2011) described two popular quality instruments. Quality Matters and Sloan 
Consortium rubrics, as tools to assist institutions with measuring online quality. Quality Matters 
provides institutions with an evidence-based and peer-reviewed evaluation of online and hybrid 
course organizational design, but does not determine effectiveness with regard to learning 
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interactions, course delivery, or institutional support for online courses and programs (Battin- 
Little, 2009; Pollacia & McCallister, 2009; Westerfelt, 2011). The Sloan Consortium instrument 
provides a framework for determining online quality at the institution level, but does not evaluate 
course delivery practices (Bourne et al., 2005; Kee Meng & Mayadas, 2010; Westerfelt, 2011). 

One criticism of online program evaluation instruments is the lack of flexibility 
traditionally associated with narrative-style assessments used in face-to-face learning (Forsyth et 
al., 2010). Instruments for distance learning often limit assessors to rigid structures and 
evaluation standards that frequently control the delivery and evaluation of online learning 
(Forsyth et al., 2010). Distance education researchers have advised that course development, 
delivery, and evaluation have often been the responsibility of faculty, and quality instruments 
should provide flexibility and freedom for the faculty to create and conduct courses in an online 
environment while providing guidance about quality practices (Forsyth et al., 2010; Picciano, 
2009; P. S. Smith, 2011; Wickersham & McElhany, 2010). D’Alessio and Avolio (2011) and 
Forsyth et al. (2010) further described faculty expertise in the creation of academically sound 
and high quality courses and programs. Some scholars have suggested that an instrument that 
guides quality in distance courses should combine the ability to support and monitor the essential 
elements of course quality, flexibility, and academic freedom with faculty support for continual 
improvement and skills development for conducting high quality programs (Picciano, 2009; 
Wickersham & McElhany, 2010). 

Statement of the Problem 

Middle States Commission for Higher Education, the United States Department of 
Education, and New York State Department of Higher Education require that higher education 
institutions establish and monitor quality with regard to post-secondary, distance-based, degree 
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granting, programs in a discipline agnostic approach rather than prescribing one set of guidelines 
for programs (Middle States Commission on Higher Education, 2011, New York State Education 
Department Office of College and University Evaluation, 2011). Distance education scholars 
proffer that administrators and faculty continue to indicate concern with distance education 
course quality throughout the various disciplines and departments within their respective 
institutions (Austin, 2010; Endean et al., 2010; Forsyth et al., 2010; Hoskins, 2009; Kee Meng & 
Mayadas, 2010; Picciano, 2009; P. S. Smith, 2011; Westerfelt, 2011; Wickersham & McElhany, 

2010) . The problem is that quality evaluation processes for distance education do not account 
for the complex differences in pedagogical approaches and instructional delivery across 
disciplines or institutions (Endean et al., 2010; Forsyth et al., 2010; Picciano, 2009; Postek et al., 
2010; P. S. Smith, 2011; Westerfelt, 2011). 

Magdalene University has slowly grown the number of distance education courses from 9 
in the 2002-03 academic year to 55 in the 2009-10 academic year (P. L. Beaman, personal 
communication, April 10, 2012). Administrators are seeking to increase the number of distance 
education courses to 25% of all courses offered (M. S. Brogan, personal communication, July 22, 

2011) . Magdalene University faculty express concern that institutional goals for the expansion 
of distance education are not associated with adequate support or attention to issues of quality 
(Graham & Jones, 2011; LaPrade, Marks, Gilpatrick, Smith, & Beazley, 2011; Lee et al., 2010; 
Singleton & Session, 2011). Magdalene faculty members desire a system that supports 
accomplishing the distance education expansion goal while maintaining academic quality and 
rigor and supporting faculty development of distance education courses (M. S. Brogan, personal 
communication, July 22, 2011). Magdalene University is also initiating a fully online graduate 
program and have a need, under both New York State higher education regulations and Middle 
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States Commission on Higher Education, to have quality guidelines, tools, and support in place 
prior to enrolling students (Middle States Commission on Higher Education, 2011, New York 
State Education Department Office of College and University Evaluation, 2011). 

The study follows the Formative Research method’s constructivist and collaborative 
nature to provide evidence for the iterative revisions and final acceptance of the quality course 
design framework. The method respects scholarly recommendations that issues of curricular 
quality are best investigated and solved by interdisciplinary collaboration among interested 
parties including faculty and instructional designers (Pratasavitskaya & Stensaker, 2010). 

The population of this study was made up of faculty members from the small liberal arts 
university where the study took place. Participants were encouraged to participate by following 
usual communication channels including email requests. Study participants came from a wide 
range of academic disciplines with both tenured and non-tenured faculty, both part time and full 
time, and have a variety of experiences teaching distance education courses. 

Purpose of the Study 

The purpose of the Formative Research study was to develop an instructional delivery 
framework for distance education courses at a small private liberal arts university in western 
New York state. The framework meets requirements from accreditation and regulatory agencies, 
faculty and administration concerns about how to design and implement quality distance courses, 
and maintenance of pedagogical flexibility for faculty. Instructional delivery items were well- 
researched, evidence-based, and institutionally sound to provide adequate guidance to 
experienced and inexperienced distance education faculty. 

Selection of ten study participants was purposive to include both full and part time 
professors and instructors from a range of disciplines and those who have and have not 
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experienced teaching distance learning classes. Data collection techniques included using 
observations of study participants, semi-structured interviews (see Appendix D), and focus 
groups (see Appendix E). Accurate transcription, member checking, and triangulation 
eliminated researcher bias in the data collected (Lichtman, 2013). 

Significance of the Problem 

Significance of the study. The Formative Research study is significant because the study 
develops a course delivery framework that addresses design and pedagogical techniques that 
allow for flexible, evidence-based, and supported instruction of distance courses at the 
institution. A literature review revealed an incomplete and unbalanced body of knowledge about 
quality of distance education programs, which could be addressed with an adequate course 
delivery framework that provided guidance to faculty designing and teaching online courses. 

The framework also supports departments and administration in reporting quality delivery of 
distance courses for institutional assessment and accreditation purposes. Determining and 
supporting quality in distance education course delivery is difficult because existing models such 
as Sloan-C Five Pillars of Quality in Distance Teaming and Quality Matters do not address the 
pedagogical delivery of such courses (Battin-Fittle, 2009; Bourne et al., 2005; Endean et al., 
2010; Forsyth et al., 2010; Kee Meng & Mayadas, 2010; Picciano, 2009; Pollacia & McCallister, 
2009; Postek et al., 2010; P. S. Smith, 2011; Westerfelt, 2011). The literature reveals that a lack 
pedagogical and technological skills exist with which to deliver effective instruction due to the 
numerous and changing techniques, tools, and practices in online courses among faculty (Al- 
Salman, 2011; Daukilas, Kaciniene, Vainoriene, & Vascila, 2008; Graham & Jones, 2011; Hae- 
Deok, Wei-Tsong, & Chao-Yueh, 2011; Kupczynski, Mundy, & Maxwell, 2012; FaPrade et al., 
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2011; Lee et al., 2010; Orr, Williams, & Pennington, 2009; Sarrico, Rosa, Teixeira, & Cardoso, 
2010; Singleton & Session, 2011; Tabata & Johnsrud, 2008). 

Significance of the study to leadership. The gap in literature regarding the quality of 
distance education assessment processes poses risks for the academic and administrative 
leadership of higher education due to the increasing external demands to deliver and report the 
quality of distance learning programs and courses (Daniel, Kanwar, & Uvali-Trumbi, 2006; 
Gaytan, 2009; Jung, Wong, Chen, Baigaltugs, & Belawati, 2011; Peinovich, 2008; Sener, 2010; 
Seok, 2007). Addressing this breach in distance education quality requires the development of 
institutionally specific systems and processes and models that integrate quality into the delivery 
of courses (Al-Salman, 2011; Forsyth et al., 2010; Graham & Jones, 2011; Hae-Deok et al., 

2011; Huett, Moller, Foshay, & Coleman, 2008; Kupczynski et al., 2012; Lee et al., 2010; 
Orellana, 2006; Orr et al., 2009; Postek et al., 2010; Schuck, Gordon, & Buchanan, 2008; 
Singleton & Session, 2011; Tabata & Johnsrud, 2008). The Formative Research study provides 
higher education leadership both a framework to deliver quality courses and a model to 
implement within their individual institutions to support institutional assessment and 
accreditation processes. 

Nature of the Study 

Overview of the research method. To support the faculty of Magdalene University and 
prepare the institution for the expansion of distance education courses and programs, the study 
followed a Formative Research method to design a new quality course delivery framework. 
Formative Research is a qualitative case study-based research method appropriate for use with 
the development or improvement of an instructional design theory, model, or framework 
(Reigeluth & Frick, 1999). The method is constructivist in nature and uses collaborations among 
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interested parties to create or redesign instructional design related models, theories, or 
frameworks (Reigeluth & Frick, 1999). The research followed a five-step process: 

1 . Create a case in supporting the new instructional design framework. 

2. Collect and analyze formative data. 

3. Revise the instructional design framework. 

4. Repeat steps two and three until the research reaches a point of saturation. 

5. Fully develop the tentative instructional design framework. (Reigeluth & Frick, 

1999; Reigeluth, 2009) 

The fourth step of the research processes repeats within the cycle until the research 
reaches the point of saturation where no significant changes or improvements are suggested by 
the study participants (Reigeluth & Frick, 1999). Study participants receive an introduction to 
the quality framework for designing and delivering distance courses along with a case example 
applicable to the framework. Data were collected by observation of participants, design of a 
course with the framework, individual interviews (Appendix D), and focus group interviews 
(Appendix E). Data were transcribed and analyzed leading to revisions of the model. This 
process repeated until the study reaches saturation and no more revisions were deemed valuable 
or sought from participants and the framework was finalized (Appendix F). 

Overview of the design appropriateness. The Formative Research study method is 
implemented with a group of participants in a collaborative experience to find and develop 
enhancements of instructional design theories, models, or frameworks (Reigeluth & Frick, 1999). 
The method is appropriately applied to curricular quality in higher education. Instructional 
quality is improved when instructors and supporting administration collaborate on solving issues 
related to course quality (Daukilas et al., 2008; Jordens & Zepke, 2009). Specific to distance 
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education, scholars concur that collaborations between administrators and faculty on distance 
learning quality lead to superior results for the institution, students, and faculty (Gaytan, 2009; 
Jordens & Zepke, 2009; Seok, 2007). Formative Research is an established method in education 
when it is applied to system improvements (Doblar, 2010; Schankman, 2006), instructional 
technology and design (Enfield, 2012; Hsu, 2009; Roskos, Burstein, You, Brueck, & O’Brien, 
2011; Squire, 2008; F. Wang & Hannafin, 2005) and specifically to issues of instructional design 
in distance learning (Halverson, 2006; Snyder, 2006; Yagodzinski, 2012). 

Research Questions 

Research questions help focus the study and guide the research as it progresses (Denzin 
& Lincoln, 2011). The goals of a Formative Research study with the purpose of designing a new 
instructional design framework are to meet the demands for appeal, ease of use or efficiency, and 
effectiveness (Reigeluth & Frick, 1999). This Formative Research study was an effort to explore 
the following research questions: 

RQ1: What are the perceptions and attitudes of faculty in a small university about how a 
proposed distance education quality course delivery framework is appropriate and appealing to 
them? 

RQ2: What are the perceptions and attitudes of faculty in a small university about how a 
proposed distance education quality course delivery framework will meet their needs for 
effectiveness in delivering quality online courses? 

RQ3: What are the perceptions and attitudes of faculty in a small university about how a 
proposed framework will meet their needs for ease of use? 
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Definition of Term s 

The following definition of terms provides an understanding of concepts and notions used 
throughout the Formative Research Case Study. These terms were selected based on the 
relevance of the term to the study. For the purposes of the present study, the following 
definitions assist with clarity, context, and understanding. 

Academic auditing. Academic auditing is investigation regarding educational practices 
in an institution of higher learning (Dill, 2010). 

Academic quality. Academic quality is a multifaceted concept in post secondary 
learning that demonstrates the academic integrity and rigor possible in a college or university and 
is determined through regulatory, accrediting, administration, and faculty expectations (Harvey 
& Green, 1993). 

Course delivery. Course delivery is the conveyance of instructional content, activities, 
and learning assessments to students (Dill, 2010; Filippakou, 2011; Pratasavitskaya & Stensaker, 
2010 ). 

Course logistics. Course logistics are the class management aspects of course design 
including class size, statements of responsibility, computing technology, accessibility to class 
(Orellana, 2006). 

Educational quality. Education quality is a multifaceted concept in post secondary 
learning that demonstrates the academic integrity and rigor possible in a college or university and 
is determined through regulatory, accrediting, administration, and faculty expectations (Harvey 


& Green, 1993). 
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Faculty-student interactions. These are typically collaborations between teachers and 
learners that support accomplishing course objectives (Graham & Jones, 2011; Singleton & 
Session, 2011). 

Globalization. As used in higher education, globalization is the multinational expansion 
of institutions of higher education (Daniel et al., 2006). 

Higher education quality. This is a multifaceted concept in post secondary learning that 
demonstrates the academic integrity and rigor possible in a college or university and is 
determined through regulatory, accrediting, administration, and faculty expectations (Harvey & 
Green, 1993). 

Infrastructure. Infrastructure is the set of systems, physical resources, technological 
resources, personnel, policies, and practices of a college or university (Khan & Iqbal, 2011). 

Instructional activities. These are the actions and events in a course between faculty 
and students (Daukilas et al., 2008; S. A. Lei & Gupta, 2010). 

Instructional delivery. Delivery is the conveyance of instructional content, activities, 
and learning assessments to students (Dill, 2010; Filippakou, 2011; Pratasavitskaya & Stensaker, 
2010 ). 

Learning activities. These activities are the in-class actions that support understanding 
of the content and accomplishment of the course objectives (Bers, 2008). 

Modality. Modality is a manner of teaching and learning, for example, distance 
education, face-to-face, or hybrid learning (Peinovich, 2008). 

Pedagogical flexibility. This term is defined as the flexibility of faculty members to 
select and implement the correct practice, technique, or activity into the course as deemed 


appropriate (Peinovich, 2008). 
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Scaffold. A scaffold is a logical ordering or instructional items where the former 
provides support for the latter (Ascough, 2011) 

Student-student interactions. Collaborations between learners in a class that promote 
accomplishing assignments, activities, and course objectives are termed student-to -student 
interactions (Graham & Jones, 2011; Singleton & Session, 2011). 

Assumptions 

The puipose of this Formative Research study was to develop an instructional delivery 
framework for distance education courses at a small private liberal arts university in western 
New York state. The framework meets requirements from accreditation and regulatory agencies, 
faculty and administration concerns about how to design and implement quality distance courses, 
and maintenance of pedagogical flexibility for faculty. Instructional delivery items were well- 
researched, evidence-based and institutionally sound to provide adequate guidance to 
experienced and inexperienced distance education faculty. Data were collected using 
observations of study participants, semi-structured interviews, and focus groups. 

The framework allows faculty to determine the right design and pedagogical techniques 
to address course objectives while supporting their choices with evidence-based suggested 
practices. The assumptions for the study involved an understanding of the nature and issues 
surrounding distance education and instruction of online courses. Faithful transcription, member 
checking, and triangulation were used to eliminate researcher bias in the data collected 
(Lichtman, 2013). Qualitative research requires the expertise and influence of the principle 
investigator to more completely understand the application and reality of the study (Lichtman, 


2013; Reigeluth & Frick, 1999). 
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It was assumed that there are multiple facets, perceptions, and definitions associated with 
quality in higher education and distance education and no singular standard approach would be 
most accurate (Filippakou, 2011; Harvey & Green, 1993; Sarrico et al., 2010). To address this 
concern, participants were instructed to create the sample case as they would normally develop a 
course for their discipline or individual preference. Study participants were guided to apply any 
standard of quality or rigor that would be common to their department or discipline. 

The study included an assumption that faculty members could deliver quality distance 
education courses if properly supported through a systematic framework that provided evidence- 
based pedagogical techniques (Al-Salman, 2011; Daukilas et al., 2008; Graham & Jones, 2011; 
Hae-Deok et al., 2011; Harvey & Green, 1993; Kupczynski et al., 2012; LaPrade et al., 2011; Orr 
et al., 2009; Sarrico et al., 2010; Singleton & Session, 2011; Tabata & Johnsrud, 2008). This 
assumption was addressed by providing participants instructions about using the sample case and 
instructional delivery framework. Participants were free to ask questions about using the 
instrument and seek guidance and support as needed. 

Finally, it was assumed that addressing delivery of distance education courses affects 
overall programmatic and institutional academic quality (Al-Salman, 2011; Forsyth et al., 2010; 
Graham & Jones, 2011; Hae-Deok et al., 2011; Huett et al., 2008; Kupczynski et al., 2012; Lee et 
al., 2010; Orellana, 2006; Orr et al., 2009; Postek et al., 2010; Schuck et al., 2008; Singleton & 
Session, 2011; Tabata & Johnsrud, 2008). Addressing this assumption, the design of the 
instructional delivery framework included course related regulations, guidelines, and best 
practices that lead to quality course delivery. The instrument included links to guidelines from 
specific disciplinary fields where faculty could seek external standards of quality related to 


course delivery. 
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Scope 

The study took place at a small, private, liberal arts four-year university in western New 
York state referred to in this study with the pseudonym Magdalene University or Magdalene. In 
2011, the institution had a full time enrollment of 3,005 students (Institute of Educational 
Sciences, N.D.). The university where the study took place employs 1 19 full time professors and 
149 part time instructors (Institute of Educational Sciences, N.D.). Selection of participants was 
purposive to include both full and part time professors and instructors from a range of disciplines 
and those who had and had not experienced teaching distance learning classes. Magdalene 
University offers undergraduate, graduate, and clinical doctorate degrees in 17 academic 
disciplines (Institute of Educational Sciences, N.D.). The study required the participation of ten 
full and part time faculty members from the Magdalene University. Research took place in a 
span of 8 weeks. 

The purpose of the Formative Research study was to develop an instructional delivery 
framework for distance education courses at a small private liberal arts university in western 
New York state. The framework meets requirements from accreditation and regulatory agencies, 
faculty and administration concerns about how to design and implement quality distance courses, 
and maintenance of pedagogical flexibility for faculty. Instructional delivery items were well- 
researched, evidence-based, and institutionally sound to provide adequate guidance to 
experienced and inexperienced distance education faculty. Data were collected using 
observations of study participants, semi-structured interviews, and focus groups. The research 
process included the development of a drafted instructional design framework, collection of data, 
transcription of data, analysis of collected data, and development of a tentative instructional 
design framework specific to the purpose of the study. 
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The scope of the study included the development of a drafted instructional design 
framework and associated case to provide study participants with a means to apply the 
framework and generate opinions and experiences. The framework and case related specifically 
to the design and delivery of a distance education course. The scope also included the 
observation of faculty members from the institutional site in the application of the instructional 
design framework. The study also included the use of individual and group interviews about the 
design experience using the instructional design framework. 

Data were collected from ten study participants consisting of faculty members from the 
research institution probed through use of researcher notes and audio recordings of observations, 
semi-structured interviews, and focus groups. Notes were transcribed accurately and analyzed for 
improvements to the instructional design framework. Collected data was used to make 
improvements to the framework and repeated the cycles of observation, data collection, analysis, 
and improvement to the framework until the study reached the point of saturation. Finally, a 
fully developed tentative instructional design framework (Appendix F) was created to support 
faculty designing and delivering distance education courses 
Limitations 

Research studies have inevitable limitations that must be addressed to mitigate issues of 
validity (Cone & Foster, 2006). The limitations of this study stemmed from the foundational 
theory used to build the Formative Research method. Formative Research is modeled after a 
single holistic case study method (Reigeluth & Frick, 1999). Yin (2009) suggested that single 
holistic case studies or Type 1 studies contain limitations in generalizability beyond the single 
case, shifting or changing case aspects during the research, and lack of sufficient rigor. 
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Formative Research addresses these limitations in the addition of research phases specific to 
instruction design concerns (Reigeluth & Frick, 1999). 

Yin’s (2009) supposition that single holistic case studies limit the generalizability of the 
research beyond the specific case situation is a concern in the present study. The limitation is 
addressed with the focus on one instructional design theory, model, or framework (Reigeluth & 
Frick, 1999). The highly individualized institutional approach to determining quality in higher 
education and distance learning provide rational for using such a model (Pratasavitskaya & 
Stensaker, 2010). 

Type 1 case studies as the foundational theory supporting Formative Research are also 
limited due to shifts and changes that occur during the research phase (Reigeluth & Frick, 1999). 
This limitation was addressed by the iteration of observation, collection, analysis, and framework 
revision stages through the research phase (Reigeluth & Frick, 1999). The use of different study 
participants whenever possible through the iteration stages increases the likelihood that most 
concerns, interests, styles, and perceptions are addressed (Reigeluth & Frick, 1999). 

Finally, single holistic case studies can lack sufficient rigor and are often criticized as the 
experiences of the principle investigator or research team chronicled throughout the situation 
(Yin, 2009). In Formative Research this limitation is addressed by using rigorous data 
collection, accurate transcriptions thorough analysis techniques, member checking, and 
triangulation to ensure that bias is minimized and results are as true to the study as possible 
(Reigeluth & Frick, 1999). 

Delimitations 

The study was delimited by the decision not to select participants, but instead rely on 
requests from institutional site leadership for volunteers. The population was delimited to those 
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that expressed an interest, had the available time to participate, and teach at Magdalene 
University. It is recommended, but not necessary, for the iterative steps of observation, data 
collection, analysis, and framework revision to use different study participants for each cycle 
whenever possible (Creswell, 2012). The potential study participant pool was limited by the 
institutional staffing as well. 

Summary 

In this chapter, the Formative Research study was introduced. The problem was stated as 
quality evaluation processes for distance education do not account for the complex differences in 
pedagogical approaches and instructional delivery across disciplines (Endean et al., 2010; 

Forsyth et al., 2010; Picciano, 2009; Postek et al., 2010; P. S. Smith, 2011; Westerfelt, 2011). 

The purpose of the Formative Research study is to develop an instructional delivery framework 
for distance education courses at a small private liberal arts university in western New York state 
referred to with the pseudonym Magdalene University. The framework meets requirements from 
accreditation and regulatory agencies, faculty, and administration concerns about how to design 
and implement quality distance courses, and maintenance of pedagogical flexibility for faculty. 
Instructional delivery items were well-researched, evidence-based, and institutionally sound to 
provide adequate guidance to both experienced and inexperienced distance education faculty. 
Data were collected using observations of study participants, semi-structured interviews, and 


focus groups. 
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Chapter 2: Review of Literature 

The previous chapter provided the problem, purpose, background, and nature of this 
Formative Research study aimed at creating a quality instructional design model appropriate for 
online course delivery. The purpose of the Formative Research study was to develop an 
instructional delivery framework for distance education courses at a small private liberal arts 
university in western New York state. The framework meets requirements from accreditation 
and regulatory agencies, faculty, and administration concerns about how to design and 
implement quality distance courses, and maintenance of pedagogical flexibility for faculty. 
Instructional delivery items were well-researched, evidence-based, and institutionally sound to 
provide adequate guidance to experienced and inexperienced distance education faculty. Data 
were collected using observations of study participants, semi- structured interviews, and focus 
groups. 

The purpose of Chapter 2 is to provide an overview of the literature related to quality in 
higher education, distance education, and online course design. Scholarly books, seminal journal 
articles, research documents, doctoral dissertations, and higher education news articles were 
reviewed through the University of Phoenix library system. Database searches included 
EBSCOhost, ERIC, Google Scholar, ProQuest, ProQuest Social Sciences, SAGE Journals, and 
ProQuest Dissertations and Theses, Dissertations and Theses at University of Phoenix. Internet 
searches through Google, Bing, and Yahoo provided information related to the research study. 
Bibliographic and reference listings were accessed from appropriate titles discovered within the 
review process. Approximately 425 current scholarly publications concerning higher education 
quality, academic quality, educational quality, distance education, distance education quality, 
distance education quality instruments, online education, online education quality, distance 
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learning, distance learning quality, pedagogical quality, teaching quality, pedagogical flexibility, 
faculty concerns, administrative concerns, student concerns, institutional assessment, 
accreditation, course design, instructional design, instructional delivery, course delivery, 
teaching, and learning were reviewed and 154 scholarly resources were used in this study. 

Quality in Higher Education 

The need for the development of a quality instructional design model for distance 
education courses requires investigating what is meant by higher education quality, the public 
concerns surrounding quality in higher education, and benefits of quality to the institution. 
Increasing external pressure from parents, students, and government agencies signals a shift from 
discussing whether or not institutions should implement quality management processes to 
internal discussions about how quality should be ensured in academic content and course 
delivery (Dill, 2010; Filippakou, 2011; Pratasavitskaya & Stensaker, 2010). Quality in higher 
education consists of such factors as rising tuition costs, increased globalization of higher 
education programs (Baird, 2009; Daniel et al., 2006; Daunoriene, 2011; Singh, 2010; 
Southerland, Merlo, Robinson, Benekos, & Albanese, 2007) and political disparities between 
what is asked of quality and what institutions provide (Brink, 2010). This section of the chapter 
provides an overview of major definitions of quality, external factors influencing higher 
education quality, internal factors of quality in higher learning, and institutional assessment 
issues that lie at the center of demonstrating quality for colleges and universities. 

Defining quality. The definition of quality in higher education varies by different 
stakeholders and individuals associated with the industry (Filippakou, 2011; Harvey & Green, 
1993; Sarrico et al., 2010). Generally, the definition of quality in higher education is relative to 
the processes or outcomes associated with that person, group, or organization (Filippakou, 2011; 
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Harvey & Green, 1993; Sarrico et al., 2010). Harvey and Green (1993) argued that quality has 
no absolute definition and instead is relative to groups concerned. Higher education quality is an 
interrelated network of concerns in the areas of academics, administration, instruction, models, 
and approaches (Filippakou, 2011). Because numerous stakeholders define quality differently 
stakeholders need to better define and understand the idea of academic quality (Filippakou, 2011; 
Harvey & Green, 1993; Sarrico et al., 2010). Further, institutions and stakeholder groups must 
realize that there are varying approaches to educational quality and should attempt to alternate 
between definitions and models dependent on the audience of the conversation (Filippakou, 
2011). Harvey and Green (1993) in a widely cited report defined higher education quality with 
six major philosophies: Exception, Perfection, Fitness for Purpose, Driven by Mission, Value for 
Money, and Transformative. 

Exception. Closely associated with the traditional thinking about quality in that 
institutions of high value are distinctive and unique from others is the Exception philosophy of 
higher education quality (Harvey & Green, 1993; Singh, 2010). Some scholars have suggested 
that the Exception philosophy is akin to elitism and not determined by assessment practices but 
rather an assumption that distinctiveness and inaccessibility together define quality (Brink, 2010; 
Harvey & Green, 1993; Jung et al., 2011). Harvey and Green (1993) suggested that the 
Exception model of quality is associated with a dialogue of excellence where the institution 
proclaims that the right things are done and better than competing schools. In many cases, this 
model demands that a set of standards is set internally, but systems of quality monitoring and 
reporting are often hidden, protected, or obscured from easy review (Brink, 2010; Harvey & 


Green, 1993). 
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Despite the obscurity of quality measurement and reporting, the Exception philosophy 
does adhere to conformance to a set of static standards that it holds are the same for all 
institutions, ignoring potential differences that may exist (Brink, 2010; Harvey & Green, 1993; 
Jung et al., 2011). In this philosophy of educational quality, it is also commonly understood that 
checking standards is the only way to showcase the quality of the institution (Brink, 2010; 
Harvey & Green, 1993). This philosophy holds that academic quality increases only when 
institutions raise standards (Brink, 2010; Harvey & Green, 1993). 

Perfection. Unlike the Exception philosophy, the Perfection philosophy of academic 
quality shifts the focus in excellence from inputs and outputs to perfections in the educational 
process and seeks to eliminate imperfections (Harvey & Green, 1993). Rather than standards to 
achieve, this model creates specifications to measure quality (Harvey & Green, 1993). This 
approach creates an atmosphere of preventing errors rather than on inspecting existing activities 
and processes for quality adherence (Harvey & Green, 1993). Everyone in the organization 
becomes responsible for monitoring, reporting, and correcting quality specifications within their 
own processes making the whole model a distributed system of adherence (Harvey & Green, 
1993). 

The Perfection model of higher education quality does allow for schools to distinguish 
themselves from others by allowing for individual institutional establishment of quality 
specifications (Harvey & Green, 1993). Within this model, even if the product or educational 
output is poor, the institution could consider it high quality provided the school followed 
established specifications (Harvey & Green, 1993). This means, as Harvey and Green (1993) 
illustrated, that researchers cannot effectively compare institutions provided the schools are 
adhering to their own individual specifications making quality determinations difficult. Harvey 
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and Green (1993) concluded that higher education is not necessarily about meeting 
specifications, but about producing a high quality graduate that other institutions would 
recognize. 

Fitness for purpose. The Fitness for Purpose approach to academic quality suggests that 
the determinant for quality is the alignment of the product or service to the purpose of the 
organization (Harvey & Green, 1993; Sarrico et al., 2010). Such a quality model is inclusive of 
all stakeholders and seeks input on both the purpose and fitness of the educational product or 
service to that established purpose (Harvey & Green, 1993). An academic product is considered 
low quality if it fails to meet the collectively established purpose (Harvey & Green, 1993; 

Sarrico et al., 2010). 

The expectations of the customers of higher education often guide this approach to 
quality (Harvey & Green, 1993). Institutions following this model, in whole or part, must 
monitor and reevaluate academics frequently to determine if the customer expectations are met 
(Harvey & Green, 1993). The challenge in adhering to customer expectations is three-fold. First, 
trends are often fluid and difficult to establish for long periods, making quality approaches 
difficult to adhere to or understand (Harvey & Green, 1993). Second, as Harvey and Green 
(1993) suggested, people unfamiliar with higher education might base expectations in an 
unachievable reality, on false beliefs, or born from misleading or incomplete marketing and 
media. Third, industries like higher education have several interrelated stakeholders making it 
difficult to know who the customers of education are or who among them has the priority 
(Harvey & Green, 1993). 

Driven by mission. This approach determines educational quality by the fulfillment of the 
institutional mission (Harvey & Green, 1993; Martinez- Argelles, Castan, & Juan, 2010). The 
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institutions’ viability in the region determines the appropriateness of the mission and 
achievement of quality (Harvey & Green, 1993). This is a common philosophy in U.S. 
institutions of higher learning as it aligns well with assessment and accreditation practices 
(Harvey & Green, 1993). This philosophy of higher education quality is not an adherence to 
standards or specification but rather establishes processes and management practices by which 
the school ensures that the desired and appropriate academic quality (Harvey & Green, 1993). 

Martinez-Argelles et al. (2010) suggested that to be successful in an increasingly 
competitive market, institutions must align quality initiatives to student satisfaction. Harvey and 
Williams (2010) concurred suggesting that with more choices in higher education the emphasis 
on quality has shifted from the supply side to the demand side and caters directly to increasing 
the number of students through satisfaction scores. Scholars caution that the danger of this 
model is that sole reliance on student satisfaction is incorrect and leads to reduced quality 
(Harvey & Green, 1993; Harvey, 2008; Schuck et al., 2008). Quality systems that heavily rely 
on student satisfaction create both informal and formal monitoring systems making it difficult to 
assess and determine quality accurately (Harvey & Green, 1993). Reliance on student 
satisfaction as a mark of quality risks falling academic standards to meet student learning 
expectations (Brink, 2010). The fallacy of this model is the notion that students might 
understand short-term needs, but lack a complete picture of quality and institutional mission, 
thus forcing rapid and dangerous shifts in the delivery of education (Harvey & Green, 1993). 
These rapid changes potential fail at aligning the academic content and quality to the mission of 
the institution (Harvey & Green, 1993). 

As an element of determining teaching and course quality student satisfaction can play an 
important role to reflective practioners desiring to make improvements or provide a better 
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learning experience for students (Bie & Meng, 2009). This shift in factoring student satisfaction 
into the determination of course quality indicates a pedagogical shift from learners as passive to 
active stakeholders in higher education (Bie & Meng, 2009). Bie and Meng (2009) caution 
students sometimes do not rate their sole satisfaction, but feel compelled to rate the quality of 
instruction on behalf of others in the class. Despite the active role students play in assessing 
instructional quality, and similarly to academic quality mentioned above, the factor should not be 
the exclusive means to report quality (Bie & Meng, 2009). 

Value for money. Subscribers to the Value for Money philosophy of higher educational 
quality determine quality as a sole factor of cost (Brink, 2010; Harvey & Green, 1993; Jung et 
al., 2011; Sarrico et al., 2010). Those higher education stakeholders following this model often 
ignore past academic performance in favor of cost of education (Brink, 2010; Harvey & Green, 
1993). Quality is achieved by continually decreasing the cost of education (Brink, 2010; Harvey 
& Green, 1993). This model is becoming popular in U.S. institutions (Harvey & Green, 1993) 
creating political pressure for administrations are reducing educational services and availability 
to students, thus lowering the quality of education as defined by other philosophies 
(Bandyopadhyay & Lichtman, 2007). 

The Value for Money philosophy often features the creation of performance indicators 
that are sometimes crudely linked to academic quality (Harvey & Green, 1993; Jung et al., 2011; 
Sarrico et al., 2010). Some examples of performance indicators common in this philosophy are 
student- to- staff ratios, revenue indexes, public versus private funding ratios, market share, and 
examination results (Harvey & Green, 1993). Quality concerns arise when acceptable 
parameters for established cost indicators fall out of compliance (Jung et al., 2011). Harvey and 
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Green, (1993) cautioned that quality becomes a concern, too, when these indicators fail to 
account or consider other quality factors not associated with cost. 

Transformative. Higher education quality in the Transformative approach centers on the 
idea that institutions evolve over time and quality only exists when institutions are changing 
(Harvey & Green, 1993; Houston, 2008; Jung et al., 2011). Harvey and Green (1993) suggested 
there are two forms of Transformation: enhancement and empowerment. The enhancement 
approach measures quality in respect to the change in students through the educational process 
(Harvey & Green, 1993). Empowerment seeks to inspire student learning through a set of 
quality practices and processes (Harvey & Green, 1993). These include use of student course 
evaluations, charters, student course selection and advisement, learning for life initiatives 
(Harvey & Green, 1993). Empowerment and enhancement approaches within the 
Transformative model of quality tend to focus on the course-related experience of students and 
institutions often adopt interdisciplinary teams designed to foster improvements in quality and 
course delivery (Jordens & Zepke, 2009). 

Harvey and Green (1993) advocated that the Transformative philosophy lie at the center 
of any higher educational quality model or conversation and suggested that the model can link 
easily to other frameworks. The model should extend beyond in-class or course-related 
experiences of students (Houston, 2008; Martinez-Argelles et al., 2010). Some scholars have 
suggested that the future of higher educational quality models should focus on a balance of 
accountability measures common in other approaches and the academic transformation of 
students and institutions (Harvey & Williams, 2010; Pratasavitskaya & Stensaker, 2010; Singh, 


2010 ). 
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External factors influencing educational quality. Numerous factors influence quality 
in higher education (Harvey & Williams, 2010). Two important factors in the U.S. are the issues 
of rising costs (Bandyopadhyay & Lichtman, 2007; Daunoriene, 2011; Ewell, 2009; Harvey & 
Green, 1993; Hersh, 2007; Z. Lei, 2009) and increasing international growth of colleges and 
universities (Baird, 2009; Daniel et al., 2006; Daunoriene, 2011; Singh, 2010; Southerland et al., 
2007). The increasing external pressure on the higher education sector evolves from the 
conversation about quality about whether or not to implement quality measures to how to 
implement those measures (Dill, 2010; Filippakou, 2011; Pratasavitskaya & Stensaker, 2010). 
This section address these two factors and provides rationale for implementing more complete 
quality models for institutions of higher learning. 

Rising costs. College-related expenses are increasing to students, families, and 
governments causing concern about the quality received from higher education and creating 
increased scrutiny of educational performance for the industry (Bandyopadhyay & Lichtman, 
2007; Daunoriene, 2011; Ewell, 2009; Harvey, 2008; Hersh, 2007; Z. Lei, 2009). Many 
stakeholders concerned with the cost of a college degree attribute quality directly to tuition costs 
and ignore the past performance of the institutions (Daunoriene, 2011; Harvey & Green, 1993). 
As a result, school administrators are dealing with popular reductions in financial support forcing 
hiring freezes, non-replacement of employees, increasing faculty loads, increasing class sizes, 
and use of lower-qualified instructional staff that, in turn, reduces quality (Bandyopadhyay & 
Lichtman, 2007). Quality in higher education is elusive, but still understood as integral to both 
student success and reporting student achievement (Daunoriene, 2011; Z. Lei, 2009). Brink 
(2010) suggested that there is a political tension between what is being asked of quality by 
stakeholders and what institutions are providing in terms of quality processes and reporting. As 
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a result, colleges and universities are pressured to provide credible and transparent evidence of 
academic achievement for students and graduates (Ewell, 2009). Francis (2010) argued that 
stakeholders need to understand higher education as both a mission to educate students and a 
business that must be responsible to ensuring success. 

The pressure from governments and citizens about the cost of higher education has given 
rise to demands for greater and more independent oversight of colleges and universities (Dill, 
2010). Dill (2010) suggested that a national agency charged with education quality needs to be 
responsible for quality measurement and reporting of student outcomes, external evaluations 
procedures, academic auditing, and comparisons of quality between schools and agencies. Ewell 
(2010) concurred suggesting that the mounting external attention on higher education is shifting 
focus away from the traditional assessment practices toward a more quantitative perspective of 
student outcomes and learning. 

Kristensen (2010) posited a counter-perspective suggesting that the creation of an 
external agency or increases in external agency procedures would only be as comprehensive as 
the institutional systems they are monitoring. Internal, institutional, quality systems remain more 
responsive to changes in the higher education landscape than a large external, governmental, 
organization (Kristensen, 2010). Brink (2010) countered this argument suggesting that the 
public is not protected from spin and is uncertain of the accountability of institutions reporting 
their own quality information. 

Increasing globalization. Another factor driving the concern with quality is the 
globalization of higher education through extended campuses and distance education programs, 
raises questions about how and how well colleges and universities in developed countries will 
offer programs in developing nations (Baird, 2009; Daniel et al., 2006; Daunoriene, 2011; Singh, 
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2010; Southerland et al., 2007). Increased marketing efforts driving the expansion of higher 
education into international markets creates more attention on issues of academic quality 
(Harvey & Williams, 2010; Southerland et al., 2007). Brink (2010) informed that global higher 
education rankings consist of academic quality and student experience in courses, which adds 
pressure to account for quality in the educational process. As such, becoming or remaining 
competitive in the global higher education marketplace requires establishing systems that 
encourage, monitor, and report on quality issues related to education (Daunoriene, 2011; Harvey 
& Green, 1993; Hersh, 2007). 

Internal factors influencing educational quality. Quality in higher education cannot be 
attained without the consideration of all interested parties and concerns (Francis, 2010). Any 
academic quality program is at risk when it represents only a narrow group of stakeholders and 
alienates the broader group of interested parties (Houston, 2010). This section provides an 
overview of internal factors that influence higher education quality in colleges and universities. 
The section is an exploration of faculty concerns, benefits to the institution, and institutional 
assessment factors. 

Faculty concerns. Faculty serve an integral role in the development and monitoring of 
quality programs and courses (Bandyopadhyay & Lichtman, 2007). Faculty often remain at a 
distance from conversations and practices of academic quality despite their importance in 
establishing and maintaining the same (Houston, 2010). Raban (2007) suggested that a culture 
of auditing academics inhibits their willingness to participate in quality processes. Raban (2007) 
continued by suggesting that the audit culture in higher education contributes to an atmosphere 
of examination and worsens the feeling of faculty disengagement from quality efforts. Another 
consequence of the audit-heavy culture in postsecondary education is the growing feeling of 
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distrust and diminished support for making quality enhancements to academics (Raban, 2007). 
Banta (2010) suggested that faculty involvement in quality matters often does not occur because 
institutions do not reward faculty for focusing on issues of academic quality. Banta (2010) 
further suggested that if institutions desire faculty involvement they should offer financial 
incentives for performance of quality practices. Administrators struggle with allocating heavy 
resource and staffing to quality efforts because infrastructures can sometimes be costly and 
returns are not often immediate (Khan & Iqbal, 2011). 

Whether the reason for faculty disengagement is financial as Banta (2010) suggested, or 
audit-rich cultures as Raban (2007) suggested, schools should create a participatory culture of 
quality that is respected by faculty (Francis, 2010). A quality culture in higher education should 
involve the collective of interested parties and groups to best develop a representative model of 
quality suitable for public understanding and consumption (Francis, 2010). The culture should 
arise from a commitment to enhance educational quality and integrate into the resource, 
budgetary, and strategic decision making of college governance (Raban, 2007). Houston (2010) 
concurred that a collaborative approach to quality is best, suggesting that establishing 
institutional level models of quality should involve all stakeholders, including faculty, and 
represent the whole of institutional thinking, rationale, and adoption of quality practices. Harvey 
and Williams (2010) also concurred suggesting that postsecondary quality is a highly political 
conversation that should include all interested parties. The risks of weak integration, as Raban 
(2007) advised, are that resources will not be allocated to monitoring academic quality. 

As a matter of public trust, Brink (2010) agreed with Houston (2010)’s supposition that 
individual institutional models are better able to meet the demands of quality. Brink (2010) 
posited that the creation of collaborative quality programs within institutions and made widely 
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available would build public trust and understanding about how quality in higher education is 
assured. This development of a collective institutional quality culture should become a 
fundamental part of how all stakeholders in the industry act and think (Banta, 2010; Harvey & 
Williams, 2010). 

Institutional growth and change. Changes in higher education arise from the 
implementation of quality assurance processes (Houston, 2010). Quality programs that account 
for failure and build in learning opportunities about those failures allow institutions to grow and 
evolve (Francis, 2010). Some scholars recommend that quality be an internal concern that 
integrates administration and faculty together, thus improving not only educational quality, but 
also the whole organization (Sarrico et ah, 2010; Schulte, 2010). These improvements in quality 
and organizational efficiency also allow the institutions to more effectively respond to changes 
imposed from external agencies and accreditors (Kristensen, 2010; Sarrico et al., 2010; Schulte, 
2010 ). 

Higher education should not detach quality from the purpose of the institution to grow 
and educate students and in turn the institution as well (Harvey & Williams, 2010). As a result 
institutions should use quality assurance methods in the areas of research, teaching, learning, and 
civic responsibility (Brink, 2010). When implemented throughout the academic process and 
programs, regardless of teaching or research focus, quality practices should aid the institution in 
strengthening those programs (Fitzpatrick, 2006). 

Institutional assessment Institutional assessment remains a steadfast method for 
colleges and universities to document and provide evidence of academic quality (Ohia, 2011). In 
conjunction with accreditation programs, institutional assessment informs schools about 
organizational strengths and weakness so the institution can plan and implement improvements 
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in quality (Goswami, Gupta, & Shukla, 2010). Brink (2010) advised that a political pressure 
exists between what external stakeholders believe about quality and what institutions are 
providing in terms of quality reporting and assurance. A tension that some scholars suggested 
exists between rituals of quality and quality as a fundamental process owned by all concerned 
parties (Banta, 2010; Harvey & Williams, 2010; Kristensen, 2010; Raban, 2007). This section 
explores institutional assessment issues that suggest the formation of more effective quality 
models for institutions of higher learning. The overview provides information about concerns 
with institutional assessment, creation of more effective quality and assessment models, and an 
increasing call to focus on course instruction. 

Trends in current assessment practices. Increasing pressure from outside higher 
education about quality shifted focus from traditional assessment methods toward a more 
quantitative emphasis on student outcomes and learning (Ewell, 2010). Current higher education 
stakeholders require that institutions report or demonstrate the return expected from the 
investment in tuition costs (Ascough, 2011). This shift confuses the notion of accountability in 
higher education with quantifiable outcome statistics (Shulman, 2007) and lends itself to the 
Value for Money philosophy of educational quality where quantifiable information drives 
decision making about quality practices like assessment (Reigeluth & Frick, 1999). The heavy 
reliance on quantifiable information encourages the creation and reporting of performance 
indicators like student satisfaction, student-to-staff ratios, and test results (Brink, 2010; Harvey 
& Williams, 2010). These sometimes crudely determined statistics do not fully explain the 
quality of the institution or academics (Brink, 2010; Harvey & Williams, 2010). The danger of 
relying greatly on such quantitative information is that institutions become tempted to 
compromise educational quality practices in favor of improving the reported number, for 
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example, student satisfaction (Brink, 2010). Quantitative measures regarding quality are not 
inherently in error, but should remain true to the original intentions of academic quality and 
balance with proven qualitative approaches that relate to the core values of the institution 
(Sarrico et al., 2010; Shulman, 2007). 

Similar to the movement toward more quantifiable quality reporting, there is 
encouragement for institutions to adopt a standardized approach to measuring higher education 
outcomes (Banta, 2010; Ewell, 2009; Hersh, 2007; Shulman, 2007). Some research suggests that 
institutions should become more similar in their services and education to favor standardization 
and ease the burden of institutional assessment (Shibli, 2009). Ohia (201 1) agreed that easing 
the burden of institutional assessment is possible with standardized processes and simplified 
models. The risk of standardizing education and assessment is that singular models do not 
adequately address institutional differences (Banta, 2010; Brink, 2010) or delve deep enough into 
what is happening in terms of educating students (Banta, 2010; Ewell, 2009; Hersh, 2007; 
Shulman, 2007). National frameworks, agencies, and standard measures threaten institutional 
assessment and academic quality by eliminating trust from the process, reporting only the easily 
comparable, and failing to address the full narrative of what is happening to provide a quality 
education to students (Harvey & Williams, 2010). 

Both the quantification and standardization of institutional quality and assessment give 
rise to the creation and growth of external agencies seeking to guide, monitor, and report on 
aspects of academic quality (Harvey & Williams, 2010; Raban, 2007). The creation of such 
external systems, while convenient as Shibli (2009) suggested, disenfranchises faculty and 
administrators who demonstrate trust in both providing and reporting educational quality 
(Harvey & Williams, 2010; Raban, 2007). Further, and similar to standardized assessment 
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practices, external agencies are incapable of understanding the full spectrum of activities aligned 
to providing a quality education (Harvey & Williams, 2010; Raban, 2007). Kristensen (2010) 
cautioned that an external system of institutional assessment in higher education would only be 
as effective as the weakest of the individual institutional quality monitoring and reporting system 
and thus risk devaluing all of education or the external quality process. Finally, despite the 
increasing focus on external assessment and accreditation processes no single agency produces a 
quality ranking suggesting that these agencies cannot fully compare and contrast individual 
institutions quality practices (Brink, 2010). 

The call for quantifiable quality reports, standardized assessment and quality practices, 
and external monitoring of higher education is a result of growing public concern to more 
transparently demonstrate academic quality (Dill, 2010; Filippakou, 2011; Pratasavitskaya & 
Stensaker, 2010). Some scholars agree that some external monitoring would support institutions 
in effectively implementing, monitoring, and reporting educational quality (Brink, 2010; Dill; 
Ewell, 2009; Ohia, 2011; Shibli, 2009). The risk is a devaluing and worsening environment for 
providing a quality education to students (Harvey & Williams, 2010). Academic quality is not 
merely an objective or standard report that is agnostic of disciplinary and institutional differences 
(Jordens & Zepke, 2009). An institutionally specific blended model of quantitative and 
qualitative quality outcomes reporting provides a complete picture of the quality delivered to 
students (Sarrico et al., 2010; Shulman, 2007). 

Need for a new institutional assessment approach. Institutional assessment is a long 
held practice for schools to provide evidence of educational quality (Ohia, 2011). The practice 
meets three needs for institutional value: continued progress toward effectiveness, pedagogical 
improvement, and accountability to external agencies (Ohia, 2011). The institutional assessment 
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process relies on subjective measures accounting for the social phenomena associated with 
education and objective measures that sometimes are misapplied to subjective criteria such as 
end of course evaluations (Juceviciene, 2009). External pressure is mounting for the creation of 
more external oversight, better transparency, more quantifiable measurements, and standardized 
approaches to assessing institutional quality (Dill, 2010; Filippakou, 2011; Pratasavitskaya & 
Stensaker, 2010). Because of this external pressure, assessment practices are sometimes force fit 
into overall models that apply results across the industry in an attempt to compare and contrast 
institutions but fail to address individual institutional differences and varying pedagogical factors 
within those institutions adequately (Pratasavitskaya & Stensaker, 2010). 

Institutional assessment is often a burden on organizations because it remains 
inadequately resourced and absent of a reliable infrastructure (Khan & Iqbal, 2011). This lack of 
resources often means rushed or delayed assessment activities when reports are scheduled or 
requested (Khan & Iqbal, 2011). Because of the external pressures to make transparent the 
quality practices of institutions of higher education, the misinterpretation or misapplication of 
assessment results, and the inadequate institutional focus on institutional assessment some 
scholars suggest the creation of more flexible, proactive, and embedded approaches to 
assessment practice within the college or university (Banta, 2010; Bers, 2008; Harvey & 
Williams, 2010; Jordens & Zepke, 2009; Juceviciene, 2009; Kristensen, 2010; Pratasavitskaya & 
Stensaker, 2010; Sarrico et al., 2010; Schuck et al., 2008; Shulman, 2007). 

Stemming from the variances in defining quality, institutional assessment also varies 
from one institution to another based upon the schools’ individual philosophies and academic 
approaches (Harvey & Williams, 2010; Juceviciene, 2009; Schuck et al., 2008). A disparity 
exists between what institutions provide in terms of assessment reporting and what concerned 
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public stakeholders are requesting from those schools (Brink, 2010). To this end, the assessment 
practice should include new and flexible approaches that more completely explain what level of 
quality is in place at any given institution (Juceviciene, 2009; Schuck et al., 2008). The flexible 
approach should include both quantitative and qualitative measures that remain true to the 
original intent of measuring quality throughout the organization from institutional to course level 
objectives (Sarrico et al., 2010). The inclusion of these measures should not alienate important 
participants in the process like faculty (Bandyopadhyay & Lichtman, 2007; Raban, 2007). It is 
understood that properly addressing educational quality is done by invested faculty and 
administrators versus non-invested personnel or outside agents (Sarrico et al., 2010). 

As an occasional practice, institutional assessment is sometimes poorly planned for which 
results in delays and frustration in the process (Khan & Iqbal, 2011). Integrating assessment into 
the resource planning, budgeting, and strategic planning, helps the institution realize greater 
benefit (Raban, 2007). Once integrated into the daily planning of the institution it embeds 
throughout processes, procedures, and practices of the institution including course instruction 
(Bers, 2008; Raban, 2007; Shulman, 2007; Singh, 2010). Updating traditional systems of 
gathering, monitoring, and reporting assessment will lessen the burden by becoming more 
streamlined (Jordens & Zepke, 2009; Ohia, 2011; Shulman, 2007). Investment in new 
technologies will make assessment less onerous on the institution, administration, and faculty, 
new technologies (Jordens & Zepke, 2009; Shulman, 2007). Finally, a new quality and 
assessment approach should provide for follow through to ensure good quality continues and 
areas in need of improvement are adequately planned for enhancement (Harvey & Williams, 


2010; Jordens & Zepke, 2009). 
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A focus on pedagogy. Increasing attention on institutional assessment practices is 
partially because of the degradation of rigor and increased leniency about quality instruction 
(Sarrico et al., 2010). To some degree, traditional assessment practices and popular external 
oversight requests fail to address the quality of teaching and learning at the course level in a 
proactive manner (Pratasavitskaya & Stensaker, 2010). Institutions should make certain that 
assessment activities include clear statements about learning objectives, student performance, 
and student assessment measures to address pedagogy in assessment more effectively (Bers, 
2008; Ziliukas & Katiliute, 2008). Daily assessment activities should link student outcomes to 
learning objectives through course activities like instructional delivery, learning activities, 
student collaboration, and student assessment (Bers, 2008; Ziliukas & Katiliute, 2008). This 
increased focus addresses faculty concerns that pedagogical quality is not adequately addressed, 
external concerns that teaching is not meeting expectations, and embeds assessment practices 
into the daily activities of institutions (Sarrico et al., 2010). 

Quality in teaching. One study demonstrated an increased focus on academic quality 
processes improved pedagogical techniques, faculty development, curricula, and student support 
in 146 institutions of higher learning within the U.S. (Banta, 2010). A focus on teaching quality 
should be evidenced by the level of instructor preparation, planning, delivery of instruction, and 
characteristics (Almadani, Reid, & Rodrigues, 2011; Ulmer, Watson, & Derby, 2007). The 
examination should review instructional elements such as course objectives, instructional 
content, structure, learning activities, course rigor, faculty support of students, and student 
assessment (Ginns, Prosser, & Barrie, 2007; Spooren, Mortelmans, & Denekens, 2007). 
Institutional level models of quality should consider this multidimensional evaluation and 
support of teaching as it is central to pedagogical quality (Ginns et al., 2007). The inclusion of 
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these factors into an instructional design model is shown to improve faculty adoption of quality 
teaching techniques and improvements in teaching and learning excellence (Ascough, 2011). 

This section provides a review of literature surrounding issues of pedagogical quality that 
highlights considerations for improving teaching quality through collaborative experiences, 
flexible instruction, and instructional quality models. 

Increasing criticism from without higher education is creating a demand for more 
standardized quality models that include teaching-level criteria (Banta, 2010; Ewell, 2009; 

Hersh, 2007; Shulman, 2007). The dangers of implementing a standardized instructional quality 
framework is that uniform modeling does not account for the variances in teaching expertise that 
are at the center of educational philosophy and does not respond to the variances in learners 
(Yair, 2008). Quality concerns from outside higher education should be carefully considered and 
included where those demands meet the goals of education (Jordens & Zepke, 2009). This can 
include the addition of quantitative measures like test results, surveys, and grade comparisons, 
heavily favored by those calling for accountability, but should also include narrative and 
explanatory discussions that provide a more complete perspective about the teaching happening 
in the course, program, department, and institution (Banta, 2010; Sarrico et al., 2010; Spooren et 
al., 2007). Such blending of external concerns, internal concerns, quantifiable data, and narrative 
explanations should also include evidence that the instructional practices are research-based 
showing quality techniques and outcomes (Oermann, 2007; Sarrico et al., 2010). 

Curriculum and teaching quality is improved when members of the community of 
practice collaborate to establish criteria and best practices for instruction (Jordens & Zepke, 
2009). Evidence suggests that this community of practice include both faculty and those that 
support quality instruction within an institution (Pratasavitskaya & Stensaker, 2010). Pozarnik 
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(2009) suggested that these inclusion of supporting roles like assessment professionals and 
instructional designers provide faculty development and support that in turn increases academic 
quality. The goal of these collaborations is the creation of interdisciplinary frameworks that 
promote quality instructional design, delivery, and evaluation (Jordens & Zepke, 2009). 

An instructional quality model should not seek to limit faculty into teaching with specific 
techniques, styles, or practices but instead leverage the collaborations of instructional personnel 
to ensure that instructional techniques are well researched and evidence based (Oermann, 2007 ; 
Pratasavitskaya & Stensaker, 2010; Yair, 2008). The ability for faculty to make pedagogical 
decisions about course design and delivery is directly linked to student success and course 
satisfaction (Mancuso, 2009; McLawhon & Cutright, 2012; Selwyn, 2011). The use of evidence 
creates a defensible, but flexible approach allows for the teaching freedom necessary to instruct 
different disciplines, topics, and learners (Pratasavitskaya & Stensaker, 2010; Yair, 2008). 

Scholars agree that exemplary faculty would reject a rigid instructional design model as 
they do not feel constrained or reverent to the such structures (Kirkwood & Price, 2008; Yair, 
2008). These faculty members report that allowances for flexibility enable them to better meet 
the needs of the student in relationship to the content and through a variety of pedagogical 
practices (Kirkwood & Price, 2008; Mancuso, 2009; Yair, 2008). As a matter of attitude toward 
teaching, faculty members report a more positive approach to instruction when they can vary 
their instructional activities to demonstrate their breadth of knowledge on both the subject area 
and teaching practice (Ching- San, Farn-Shing, Chen-Tung, Fang- Chung, & Chia-Yu, 2009; 
Mustafa & Dalen, 2006; Yair, 2008). 

The need for flexibility in a quality instructional design framework echoes student 
concerns as well. There is an increasing shift from traditionally lecture-rich instruction toward 
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more flexible and varied student-centric approaches that include active learning, multiple 
methods of delivery, goal -based learning, and increased student feedback (Pozarnik, 2009). 
Students agree that quality instruction is critical to their success and that quality is directly 
attributed to educational experiences that flex with the needs of the course and students (Ahmad 
& Bahi, 2010). The ability for a teaching professor to vary the instruction in a course creates a 
learning environment that increases student success and performance (Ascough, 2011; Mustafa 
& Dalen, 2006; Yair, 2008). In terms of student ratings of faculty members, high performing 
students rated their experience higher when the faculty member exhibited a high degree of 
autonomy and flexibility with the course instruction (Mustafa & Dalen, 2006) 

In sum, creation of a quality framework in support of pedagogical excellence meets the 
needs of external critics and concerned parties while allowing for a teaching-centric 
collaboration. Such a model would not only support high quality instruction directly (Jordens & 
Zepke, 2009) but also provide faculty development in the areas of quality course design and 
instruction (Pozarnik, 2009). Finally a quality process for constructing and teaching courses 
provides faculty, departments, and institution a reflective process that is supported by evidence 
and meets the needs of a wide range of stakeholders (Sarrico et al., 2010). As such, a course- 
level quality framework enables the faculty and administration to easily build in assessment and 
quality monitoring practices and support improvements in quality, confidence, and value to 
students (Bandyopadhyay & Lichtman, 2007). 

Quality in Distance Education 

Distance education provides a mechanism for colleges and universities to expand 
internationally, provide broader access to potential students, and increase enrollments in existing 
or new programs (Daniel et al., 2006; Jung et al., 2011; Seok, 2007; Singh, 2010). One indicator 
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of growth in higher education is student enrollment; student enrollment in online courses 
exceeded 6.1 million students in the Fall 2010 semester, a growth of over one half million 
enrollments from the prior year and exceeded the growth of face-to-face course enrollment in the 
same period (Allen & Seaman, 2011). The growth indicates significant demand for this modality 
of learning (Daniel et al., 2006; Jung et al., 2011; Peinovich, 2008; Seok, 2007). Despite the 
intensification of online learning, the modality has failed to eclipse traditional higher education 
(Peinovich, 2008). Instead, distance learning has become a viable option for some students and 
faculty as it is offered in a diverse landscape of instructional modalities (Peinovich, 2008). 

The emergence of distance education as an option in traditionally face-to-face institutions 
came largely from the impetus of a few interested faculty members or as a specific mandate from 
school administrations (Gaytan, 2009). Concerns about quality persist and are fueled by 
expansive growth in online student enrollments (Daniel et al., 2006; Jung et al., 2011; Peinovich, 
2008; Seok, 2007). Peinovich (2008) posited that concerns specifically about quality in distance 
education are an important aspect of the greater public concern about higher education quality. 

A complicating factor in the discussion about online quality is the disparity in understanding and 
consensus about higher education quality in general (Harvey & Williams, 2010; Tanner, Noser, 

& Totaro, 2009b). 

Some scholars advocate for well established institutional processes including a quality 
framework and policy reviews to assure the public that quality exists, students are protected, and 
the public investment in education is safeguarded against low quality online courses (Jung et al., 
2011; Sener, 2010). Quality in distance education should focus on nine areas: access, learning 
interactions, technical support, curriculum and instruction, evaluation and assessment, 
institutional commitment, infrastructure and budget, and learning resources (Gaytan, 2009). 
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Some researchers suggest specifically designing quality protocols and guidelines for online 
course delivery that encompasses course design, instructional development, delivery, and 
pedagogical strategies used to conduct online courses (Chien, Durrington, & Olinzock, 2004; 
Gujar & Vadnere, 2011). Those people involved with distance education are becoming more 
learned about what constitutes quality in distance education (Betts, Hartman, & Oxholm, 2009). 
Online course design quality becomes a central issue to the retention and persistence of students 
enrolled in distance education courses and programs (Betts et al., 2009; Chien et al., 2004). This 
section of the literature review provides information related to quality in distance education and 
covers aspects of faculty concerns, administrative concerns, regulatory and accreditation issues, 
current quality models, and instructional elements related to online course development and 
delivery. 

Faculty perspective. Faculty members contribute directly to distance education in terms 
of development and instruction making it important to understand their perspectives about the 
quality of online course delivery (Graham & Jones, 2011; Tanner et al., 2009b). Faculty 
perception of distance education does is influenced by benefits for faculty (S. A. Lei & Gupta, 
2010) and concerns about the pedagogy and support present in online instruction (Singleton & 
Session, 2011; Tabata & Johnsrud, 2008). Faculty concerns with distance learning evolve from 
personal apprehensions relating to a lack of awareness regarding tasks concerned with effectively 
managing online courses and pedagogical concerns about the impact on student learning 
(Graham & Jones, 2011; Hae-Deok et al., 2011). 

Benefits of distance education to faculty. Distance education faculty members can 
realize benefits from a good experience teaching online. Most importantly, a well-constructed 
and delivered online course can result in learning gains for students (S. A. Lei & Gupta, 2010). 
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These learning gains not only apply to knowledge about course content but also the use of 
computing technology providing an added benefit post-graduation (S. A. Lei & Gupta, 2010). 

An additional benefit to faculty, and students, is the flexibility of online instruction and its ability 
to promote an easier work-life balance (S. A. Lei & Gupta, 2010). Similarly, distance education 
provides greater access to students thus extending the reach of the education made possible by 
the faculty member (Graham & Jones, 2011; Picciano, 2009) 

Some faculty argue that the nature of an always-on Internet course creates the false 
perception of being available all day every day (Schulte, 2010). The ease of access and continual 
availability makes interacting with students less cumbersome than traditional office hours 
(Schulte, 2010). Schulte (2010) contends that effective online faculty members establish 
parameters and expectations with their students promoting not only a proper work-life balance 
for the faculty member, but also the students. 

Support and necessary skills. Faculty members often do not have the initial or 
continuing training necessary to teach effectively at a distance (Graham & Jones, 2011; LaPrade 
et al., 2011; Lee et al., 2010; Singleton & Session, 2011). A lack of familiarity amongst faculty 
with the necessary computing technology, poor past experiences with teaching in the modality, 
and anxiety about using computers can serve as barriers to quality distance education (Singleton 
& Session, 2011; Tabata & Johnsrud, 2008). Research suggests that faculty members familiar 
with technology and believe that it is critical to the profession of teaching are more likely to 
effectively participate in distance education (Tabata & Johnsrud, 2008). Some scholars contend 
that the creation of faculty development opportunities and support systems for faculty around the 
use of technology and teaching at a distance are critical to success (Al-Salman, 2011; Graham & 
Jones, 2011; Hae-Deok et al., 2011; Lee et al., 2010; Orr et al., 2009). Concomitantly, faculty 
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members express a desire to participate in skills development to improve their ability to instruct 
online courses (Al-Salman, 2011; Graham & Jones, 2011; Lee et al., 2010). 

Faculty express concern that there is an increased amount of time needed to develop and 
teach a course in a distance environment (Cook, Ley, Crawford, & Warner, 2009; Graham & 
Jones, 2011; Orellana, 2006; Orr et al., 2009; Sener, 2010; Singleton & Session, 2011). In 
addition to the general concern about increased development and instruction time is the concern 
that administration does not recognize or support the additional time necessary to teach at a 
distance (Cook et al., 2009; Graham & Jones, 2011; Orr et al., 2009; Sener, 2010; Singleton & 
Session, 2011). Both faculty and administrators understand this concern and agree that time 
investment for online courses is front-loaded and diminishes over the life of a course (Bolliger & 
Wasilik, 2009). Research is still unclear about the time required to teach a single online course 
(Van de Vord & Pogue, 2012). One aspect that makes this time investment research unclear is 
that the use of instructional time and interactions is different between distance and face-to-face 
courses (Van de Vord & Pogue, 2012). 

Administrators, it is believed, do not support distance education in terms budget and 
human resources (Graham & Jones, 2011; Singleton & Session, 2011). Some faculty contend 
that administrators do not adequately support demands for the migration of courses and teaching 
to a distance education format through financial compensation or incentive (Graham & Jones, 
2011; Singleton & Session, 2011). Some faculty argue that institutions of higher education do 
not adequately account for distance education efforts in promotion and tenure processes (Graham 
& Jones, 2011; Singleton & Session, 2011). When institutional policies and budgeting properly 
account for the use of online learning faculty engaged in distance education report a higher 
degree of satisfaction with distance education (Bolliger & Wasilik, 2009; Orr et al., 2009). 
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Pedagogical concerns. Not dissimilar to the call for a standardization of higher 
education assessment is the demand for standardizing the delivery of instruction, particularly 
online instruction (Huett et al., 2008; Peinovich, 2008). Distance education creates an 
environment that contributes to the easy risk of standardizing content and instructional delivery 
(Huett et al., 2008). Faculty must not be given a prescribed model of distance education to 
ensure a quality course (Schuck et al., 2008). There is a need for models of distance education 
that provide evidence-based approaches to implementing high quality courses online (Huett et 
al., 2008). 

Included among the pedagogical concerns that distance education creates is an 
atmosphere of isolation for students. Online students report difficulty in correctly receiving and 
interpreting course materials, for example (S. A. Lei & Gupta, 2010). Faculty express concern 
that it is difficult to monitor and be aware of student emotions and attitudes throughout the 
course creating more anxiety about isolated learning (S. A. Lei & Gupta, 2010). Research 
indicates that surveyed faculty and students both expressed a feeling that students must teach 
themselves the content without much connection to the faculty member of the course (Tanner et 
al., 2009b). Contrary research suggests that the level of interactivity is greater between faculty 
and students in an distance course (Cook et al., 2009; Orellana, 2006; Singleton & Session, 2011; 
Tanner et al., 2009b). Similarly, while faculty reported that distance courses lacked the 
important outside of class interactions, students placed little emphasis on these out-of-class 
exchanges (Tanner, Noser, & Michael, 2009a; Tanner et al., 2009b). It is unclear that isolation 
and interactivity are universal concerns but could be a factor of individual course development 


and instruction (Orellana, 2006). 
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Continuing concern about interactions includes the aspect of ease of use. Teaching 
distance courses makes impromptu discussions, immediate feedback, and in-person assistance 
more difficult than in traditional face-to-face courses (S. A. Lei & Gupta, 2010). As stated, 
faculty members express great concern with the interactivity of supporting students in and out of 
class situations, but find this difficult to accomplish in distance courses (Tanner et al., 2009b). 

As such, faculty are concerned that the desired interactivity to ensure quality takes more 
technological skill and time than is generally afforded for a single course or group of courses 
(Cook et al., 2009; Orellana, 2006; Singleton & Session, 2011). This is due, in some part, to 
faculty dissatisfaction with the computing software commonly used to deliver instruction online 
as they do not yet replicate face-to-face interaction (Graham & Jones, 2011; Singleton & 

Session, 2011). This perception of technological difficulty might be due to a lack of familiarity 
and skills development in using the software (Orellana, 2006). 

Another aspect of pedagogical concern about interaction in distance courses is the 
comparison of face-to-face exchanges with online course interactions. In teaching distance 
courses, faculty express an affinity toward face-to-face interactions and report missing such 
activity in teaching online (Tanner et al., 2009b). Students, by contrast, placed less value on 
such face-to-face interactions in the same study (Tanner et al., 2009b). School administrators 
shared a lower emphasis on this course activity than faculty and were more closely aligned to 
student perceptions (Tanner et al., 2009a). This could be due to the frequency in which faculty 
members interact with students and participate in valuable synchronous exchanges (Tanner et al., 
2009b). Still, faculty holds that the technology that enables faculty-to-student and student-to- 
student interactions does not meet the standards of synchronous face-to-face (Graham & Jones, 
2011; Singleton & Session, 2011). It is also held that faculty do not all possess the skills needed 
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to work with such technologies in a way that promotes quality distance courses and interactions 
(Al-Salman, 2011; Graham & Jones, 2011; LaPrade et al., 2011; Lee et al., 2010; Singleton & 
Session, 2011). 

A major concern among faculty teaching in a distance education environment is the 
factor of class sizes. Faculty contend that large class sizes combined with the high levels of 
interaction believed to be integral to a quality distance course creates an unnecessary burden on 
faculty (Cook et al., 2009; Orellana, 2006; Singleton & Session, 2011). Confounding this factor 
is the suggestion that administrators are concerned, in part, with the optimization of class sizes 
and faculty workloads (Mancuso, 2009; Orellana, 2006). Administrators are reported to 
sometimes hold that online courses will enable an instructor to deliver content to more students 
without consideration of the required quality components of interaction, assessment, and learning 
activity (S. A. Lei & Gupta, 2010). Anecdotal evidence suggests that smaller class sizes equate 
to a more balanced and appropriate level of interactivity (Orellana, 2006). Orellana (2006) 
researched online courses of varying size and interaction levels and found little correlated 
evidence supporting claims that smaller classes have better levels of interactivity leading to 
student success. To this end, Orellana (2006) suggested that institutions establish their own 
guidelines about class sizes, interactivity, and student success. 

The growth of distance learning creates faculty questions and concerns about the best 
methods for instructing an online course (Kupczynski et al., 2012). Faculty concerns about the 
quality teaching of distance education courses evolve from concerns due to a lack of familiarity 
with the modality, to concerns about skills and abilities in teaching in this environment, to 
concerns about the success of students (Graham & Jones, 2011; Hae-Deok et al., 2011). These 
concerns are abated with a widely supported technological infrastructure that includes support 
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for technical skills development and flexible course technologies (Graham & Jones, 2011; Hae- 
Deok et al., 2011; Orr et al., 2009). Institutions should also encourage faculty to participate in 
distance education by offering financial incentives and course load balancing (Graham & Jones, 
2011; Hae-Deok et al., 2011; Orr et al., 2009). Finally, faculty hesitation and concern can be 
lessened with faculty development and support systems that focus on course design, curricular 
quality, attainment of learning objectives, faculty-to-student interactions, and use course 
technology (Al-Salman, 2011; Chien et al., 2004; Graham & Jones, 2011; Hae-Deok et al., 2011; 
Lee et al., 2010; Orr et al., 2009). 

Administrative perspective. Higher education administrators authorize and fund 
distance education initiatives at their respective institutions making it important to understand the 
issues and concerns this group of individuals faces with regard to quality in distance learning 
(Tanner et al., 2009a). Administrators are concerned with how to expand online learning to 
remain competitive while encouraging and supporting faculty to meet the increasing demand for 
this method of learning (Gaytan, 2009; Orr et al., 2009). Higher education administrators 
maintain a careful watch over rapidly tightening budgets and demands for a growing 
infrastructure that supports effective distance education programs (S. A. Lei & Gupta, 2010). 

This section provides a review of literature that addresses the concerns of administrators in 
higher education. 

Demand and faculty encouragement. In a 2011 published report, 65% of U.S. 
institutions of higher education reported distance education was an integral part of their strategic 
planning (Allen & Seaman, 2011). To remain competitive and relevant in an increasingly 
diverse marketplace for higher education, traditional school administrators are strategically 
planning and encouraging the growth of distance education programs (Bolliger & Wasilik, 



49 


2009). In a 2009 study regarding administration perspective about online education, 85% of 
administrators surveyed reported that their college or universities would continue or increase 
offering distance education courses (Gaytan, 2009). 

Administrators are concerned with faculty interest in creating distance education as a 
viable option for students at their institutions (Hae-Deok et al., 2011). The conflict between 
administrators who desire more distance education courses and reticent faculty stems from 
concerns that the decision was made to move more courses online without adequate investigation 
into the time demands, infrastructure required, class size limits, role of the institution in the 
marketplace, and required technical literacy skills for students (Bolliger & Wasilik, 2009; 

Gaytan, 2009; Singleton & Session, 2011). Administrators are also concerned with the ability of 
faculty to instruct from a distance without the proper skills development and support systems in 
place (Gaytan, 2009; Orr et al., 2009). 

In a 2009 published study about administrator perspectives on the growth of distance 
education at their institutions, 88% of respondents encouraged the growth of online learning, but 
only 34% reported an understanding of the time demands required to develop a distance course 
and only 14% felt the time demand was greater than that of a face-to-face course (Gaytan, 2009). 
The disparity between faculty and administrators about the time demands for development and 
instruction of a distance course can serve as a barrier to moving courses to that learning modality 
(Bolliger & Wasilik, 2009; Sener, 2010). Anecdotally, it is understood that faculty who wish a 
higher degree of interaction, which is linked to high quality courses, should prepare to spend 
more time on distance courses than face-to-face sections of the same course (S. A. Lei & Gupta, 


2010 ). 
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Increased budget and resources. Higher education administrators are charged with 
authorizing and funding distance education when institutions elect to include this learning option 
into their offerings (Tanner et al., 2009a). As such, administrators are concerned with the cost 
and resourcing issues surrounding infrastructure, development, and ongoing support for distance 
education (S. A. Lei & Gupta, 2010). Administrators are also concerned with issues like 
optimizing class sizes, faculty workloads, and quality of instruction (Mancuso, 2009; Orellana, 
2006). 

For some, the lure of distance education was a method to reduce the cost of materials and 
instruction by creating easily replicated and distributed content to a wider audience (Betts et al., 
2009; S. A. Lei & Gupta, 2010). While innovations like distance education do have some ability 
to optimize the costs of education thus keeping tuition better controlled, the costs of distance 
education are often hidden in other services and departmental budgets (Betts et al., 2009). 
Institutions must factor into their distance education budget the direct and indirect costs of course 
materials development by faculty or other professionals, faculty skills development, computing 
hardware and software, technical support for students and faculty, and academic support for 
students (S. A. Lei & Gupta, 2010). When factoring in the all the costs of distance education 
including the increased costs of supporting faculty and students in a reliable ecosystem, the myth 
of reducing costs through online learning is somewhat dispelled (S. A. Lei & Gupta, 2010). 

Despite the cost reduction myth, the growth of distance education demand means 
institutions must take seriously offering this option to students (Bolliger & Wasilik, 2009). 
Schulte (2010) suggested that distance education demands from students and faculty are due, in 
part, to the flexible workflow of the modality. Schulte (2010) argued that the flexibility 
promotes a healthier work-life balance and extends the reach of education to students. Further, 
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distance learning offers a flexible format that accommodates student and faculty schedules but 
also because it alleviates pressure on overused physical plants, parking, and institutional services 
(S. A. Lei & Gupta, 2010). 

Quality concerns for administrators. Administrators, like faculty, are also concerned 
with the quality of instruction delivered through distance learning platforms (Gaytan, 2009). In a 
2009 published study regarding distance education offering from colleges and universities, 85% 
of administrator respondents indicated that they believed quality is possible for distance 
education courses (Gaytan, 2009). In the same study, administrators shared a faculty concern 
that distance courses quality was not as high as face-to-face courses due to the lack of 
interactivity between faculty and students (Gaytan, 2009). Another 2009 study indicated that 
administrators are less concerned than faculty that interaction leads to quality instruction (Tanner 
et al., 2009a). As a factor of quality, distance education technologies do not replicate face-to- 
face interaction and thus are not considered the same leading some professionals to the 
perception that interaction is not possible or as good as face-to-face classes (Graham & Jones, 
2011; Singleton & Session, 2011). 

External appeals on matters of academic quality, cost, and increased service to students 
are driving not discussions on whether or not quality should be considered but how quality 
should be implemented internally to the institutions (Dill, 2010; Filippakou, 2011; 
Pratasavitskaya & Stensaker, 2010). Distance education, particularly, requires institutions to 
establish quality measures to assure the public that the online courses meet expectations and 
organizational mission (Daniel et al., 2006; Jung et al., 2011; Peinovich, 2008; Seok, 2007). 
Kristensen (2010) suggested that the formation of internal quality systems help institutions 
remain accountable to external and internal demands as well as be more responsive to changes in 
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the environments. Administrators agree that quality is integral to the continuation and expansion 
of distance education and that it is possible with the use of quality frameworks that support 
development and implementation of online courses (Chien et al., 2004; Gaytan, 2009). A 
framework for quality distance education courses should focus on general curricular quality, 
attainment of learning objectives, faculty-to-student interactions, student-to-student interactions, 
academic integrity, and student assessment and feedback (Chien et al., 2004). 

Distance education accreditation issues. Regional accreditation agencies collaborated 
on an interregional guideline for the assessment and quality of distance programs (Hoskins, 

2009; Seok, 2007). The criteria were deliberately set broadly and left open to regional and 
institutional interpretations as to fit a quickly evolving learning modality (Seok, 2007). Seok 
(2007) indicated that the six regional commissions for the accreditation of institutions of higher 
education hold very similar or identical standards based on the interregional guidelines originally 
drafted by the regions. As such, each region must meet a similar benchmark for distance 
education courses and programs (Seok, 2007). 

The growth, change, and evolution of distance education techniques and technologies 
caused the regional accrediting bodies to evolve new guidelines that still ensure quality, but also 
allow for institutional flexibility in using new tools and achieving student learning (Qi, 2006). 
While external pressure on colleges and universities desire a standardized model of quality 
assurance, implementing a uniform framework for distance education throughout the industry 
dilutes the institutional differences and lessens the values of the institutions (Sarrico et al., 2010). 
Enforcement of a distance education standardize quality model would also provide policy makers 
and the public at risk for receiving incorrect information due to complex analysis and reporting 
procedures (Sarrico et al., 2010; Seok, 2007). 
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Continuing research with pedagogical techniques and uses of learning technology in 
distance education have encouraged regional accreditors to require member institutions to 
develop their own framework for ensuring quality (Seok, 2007). These individual frameworks 
should provide evidence-based and research-driven guidelines for conducting a class that 
illustrate quality elements of a course but allow for pedagogical flexibility within the course 
environment (Seok, 2007). An institutional framework should incorporate the experiences and 
needs of faculty members as the main delivery conduit for instructional content (Seok, 2007). 
Institutional quality frameworks for distance education should support existing accreditation, 
academic integrity, and quality processes at the college or university (Seok, 2007). Scholars 
hold that this individualized approach to accreditation and quality concerns enables institutions 
to develop effective online courses that utilize best practices for design, evidence-based 
instruction, professional development for faculty, and well-founded learning assessment 
techniques (Sarrico et al., 2010; Seok, 2007). 

In terms of accreditation, distance learning is not a separate set of guidelines, but an 
extension of the institutional quality and academic integrity guidelines already adopted by 
member institutions of the regional accrediting bodies (Middle States Commission on Higher 
Education, 2006; Seok, 2007). The guidelines for distance education are meant to assist colleges 
and universities develop institutional frameworks for quality online course delivery while 
maintaining general standards for educational quality (Seok, 2007). The Middle States 
Commission on Higher Education (2006) suggested that institutions are expected to exhibit 
distance education options to students that meet institutional standards for academic quality, 
rigor, and educational effectiveness. To this end, the regional accrediting bodies adopted the 
Best Practices for Electronically Offered Degree and Certificate Programs or Guidelines for the 
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Evaluation of Electronically Offered Degree and Certificate Programs developed by the 
Western Cooperative for Educational Technology or WCET (Middle States Commission on 
Higher Education, 2011; Seok, 2007). The standards, adopted by all six U.S. regional 
accrediting bodies, covers nine assurances of quality for distance education programs and course 
as follows: 

The Nine Hallmarks of Quality 

1 . Online learning is appropriate to the institution’s mission and purposes. 

2. The institution’s plans for developing, sustaining, and, if appropriate, 
expanding online offerings, are integrated into its regular planning and 
evaluation processes. 

3. Online learning is incorporated into the institution’s systems of governance 
and academic oversight 

4. Curricula for the institution’s online learning offerings are coherent, cohesive, 
and comparable in academic rigor to programs offered in traditional 
instructional formats. 

5. The institution evaluates the effectiveness of its online offerings, including the 
extent to which the online learning goals are achieved, and uses the results of 
its evaluations to enhance the attainment of the goals. 

6. Faculty responsible for delivering online learning curricula and evaluating the 
students’ success in achieving the online learning goals are appropriately 
qualified and effectively supported. 

7. The institution provides effective student and academic services to support 
students enrolled in online learning offerings. 
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8. The institution provides sufficient resources to support and, if appropriate, 
expand its online learning offerings. 

9. The institution assures the integrity of its online learning offerings. (Middle 
States Commission on Higher Education, 201 1, p. 3) 

Throughout the nine benchmarks of distance education quality are guidelines for the 
development and maintenance of an institutional quality framework for distance education. 
Within each standard are varying levels of quality that apply to institutional, programmatic, or 
course levels of distance education (Middle States Commission on Higher Education, 2011). 
Particular to quality course design the standard should be analyzed for course-level 
recommendations. Institutions should develop a quality model for distance education courses 
that address issues of appropriate class size, student responsibility for the course, faculty-to- 
student interactions, student-to-student interactions, learning activities, student assessment, and 
end of course evaluations (Middle States Commission on Higher Education, 2011). 

Current quality models. Development of macro-models regarding quality in distance 
education that prescriptively direct the development of online courses limits the flexibility 
faculty need to effectively convey the content in meaningful ways to students (Sarrico et al., 
2010). Such models create potential problems with institutional leadership and higher education 
policy makers because it creates difficulty discerning institutional differences in educational 
philosophy and approach (Sarrico et al., 2010). The vast array of differences between 
institutions is accounted for in the regional accreditation process regarding distance education by 
encouraging member institutions to develop their own benchmarks and frameworks of quality 


(Seok, 2007). 
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Several individual models do exist as guidelines for institutions developing or needing to 
evaluate distance education course quality (H. Wang, 2008; Y. Wang & Miller, 2006). These 
include frameworks from the National Education Association, American Distance Education 
Consortium, American federation of Teachers, Quality Matters, and Sloan-C (H. Wang, 2008; Y. 
Wang & Miller, 2006). Of the models in existence, two remain popular and most widely used: 
Quality Matters, and the Sloan-C Five Pillars of Quality in Distance Learning (Battin-Little, 
2009; Bento & White, 2010; Bourne et al., 2005; Kee Meng & Mayadas, 2010; H. Wang, 2008; 
Y. Wang & Miller, 2006; Westerfelt, 2011). 

Quality Matters model. Quality Matters is a faculty-centered and peer-reviewed model 
for organizing distance education courses (Battin-Little, 2009; Bento & White, 2010; Pollacia & 
McCallister, 2009). The Quality Matters model was developed from evidence-based research 
and applied to distance courses through a collaborative and collegial process that focuses on 
continual improvement of the course (Battin-Little, 2009; Bento & White, 2010). The model 
focuses on an evidence-based approach to eight key areas: course introduction, learning 
objectives, assessment, resources and materials, learner engagement, course technology, learner 
support, and accessibility (Battin-Little, 2009; Bento & White, 2010; Pollacia & McCallister, 
2009). Within these eight areas the rubric provides 40 individual standards of excellence for 
instructional developers and faculty to follow while developing an online course (Bento & 

White, 2010). When applied, the model requires that a team of reviewers evaluate an existing 
course and provide a rating scale with feedback from each member to the team (Battin-Little, 
2009; Bento & White, 2010). 

The Quality Matters model encourages a better alignment of the course design to the 
learning objectives of the course (Battin-Little, 2009; Bento & White, 2010; Pollacia & 
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McCallister, 2009). It enables faculty members and instructional developers to select better 
materials, clearly schedule course activities, and determine assessment devices for the course 
(Bento & White, 2010). Quality Matters also promotes clarity about course organization for 
students making it easier both to navigate and access important information in the course 
environment which can lead to higher degrees of student satisfaction and success (Bento & 
White, 2010). 

An advantage of the Quality Matters model is that it provides an easy-to-follow guide for 
faculty members to design their own online course using research-based evidence to support the 
course building (Battin-Little, 2009; Bento & White, 2010; Pollacia & McCallister, 2009). The 
model does not effectively account for the new and evolving dynamics of distance education, 
learner interactions, and new distance learning technologies (Bento & White, 2010). Likewise, 
the model does not account for faculty or developers that are unfamiliar with the various types of 
distance education technologies, tools, or techniques that can be implemented within a distance 
course (Battin-Little, 2009). The model, while effective at addressing issues of design, does not 
address the issues of course delivery (Battin-Little, 2009; Pollacia & McCallister, 2009). This 
can make it easier for novice distance education faculty to review the setup and design of the 
course but does not adequately address how effective the course was at delivering education to 
the students through various pedagogical methods (Battin-Little, 2009; Pollacia & McCallister, 
2009). A model that addresses issues of delivery and faculty development of pedagogical 
techniques as simply as Quality Matters addresses course design would better support 
institutional development of online learning, quality reviews, and continual improvement for 
distance education (Battin-Little, 2009; Pollacia & McCallister, 2009). 
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Sloan-C Five Pillars model. The Sloan Foundation began as a philanthropic foundation 
focused on improving learning outside the classroom, and with the expansion of distance 
learning naturally evolved to include initiative (Bourne et al., 2005). In advancing the awareness 
and improvements for distance learning, the foundation began a program to guide distance 
education program development for higher education (Bourne et al., 2005; H. Wang, 2008). The 
Sloan-C Five Pillars of Quality Distance Learning consists of five critical areas to guide 
institutions: learning effectiveness, access, student satisfaction, faculty satisfaction, and cost 
effectiveness or scale (Bloemer, 2009; Bourne et al., 2005; Clark, Holstrom, & Millacci, 2009; 

H. Wang, 2008). Each of these pillars is focused on the institutional development, support, and 
evaluation of distance learning programs and provides guidance in each area (Bourne et al., 

2005; H. Wang, 2008). This quality model suggests that by focusing on the stated five areas 
related to distance learning, institutions will be continually improving the quality of their online 
offerings (H. Wang, 2008). 

Learning effectiveness pillar. The learning effectiveness domain of this model evaluates 
the degree to which students completing the distance program represents the institutional goals 
for that degree (H. Wang, 2008). This assessment of learning effectiveness can take the form of 
an course development framework or post-course evaluation that examines how well the course 
design aided in achieving the course objectives (Clark et al., 2009). Likewise, institutions can 
interpret this to be an evaluation of grades and grade point averages of graduating or completing 
students (Bloemer, 2009). Determining learning effectiveness of the distance education program 
might include studying graduation rates (Bloemer, 2009). 

Access pillar. The access domain focuses on the degree to which students entering an 
online program complete the program (Clark et al., 2009; H. Wang, 2008). Institutions consider 
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the areas of academic support, administrative support, and technical support (H. Wang, 2008). 

To effectively determine the institutions progress in this area schools should consider all levels 
of academic support from entrance examinations and remedial plans to ongoing support from 
various units like the library and academic support services (Clark et al., 2009). Robust 
examinations investigate the ability for students to effectively use these services, computing 
infrastructure, and their coursework (Clark et al., 2009). To a lesser degree institutions can also 
examine the enrollment processes that enable access to the programs in general (Bloemer, 2009). 

Student satisfaction pillar. The student satisfaction domain measures the satisfaction of 
the graduates with the academic quality of the program and individual courses (Bloemer, 2009; 
H. Wang, 2008). The evaluation of this domain can include student end-of-course surveys and 
course enrollments which includes during course changes in enrollment numbers (Bloemer, 
2009). Detailed examinations can also include evaluation of the out-of-class learning and social 
environments, and national surveys of students satisfaction (Clark et al., 2009). 

Faculty satisfaction pillar. Faculty satisfaction is a domain that determines the degree to 
which the instruction meets the needs and beliefs of the faculty with regard to quality (H. Wang, 

2008) . This domain is an assessment not only of faculty contentment with course quality, but 
also the extent to which faculty members can effectively provide quality instruction (Clark et al., 

2009) . As such, evaluations of this domain include assessments of faculty development and 
faculty technical support (Bloemer, 2009; Clark et al., 2009). This domain can also determine 
effective course loads for faculty, number of full and part time faculty appointments in distance 
education, compensation, release time for developing new courses, and ongoing research in the 


field (Clark et al., 2009). 
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Cost effectiveness pillar. Cost effectiveness or scale is a domain that measures the value 
of the program compared to the overall costs to run it and continue expansion for new 
enrollments (Clark et ah, 2009; H. Wang, 2008). Determining the success of the institution in 
this area requires that schools examine distance education budgets to ensure a sustainable 
financial foundation for the program (Bloemer, 2009; Clark et al., 2009). Assessment in this 
area also requires determining enrollment projections to effectively budget for ongoing programs 
(Clark et al., 2009). A complete evaluation would consider both the direct cost of instruction and 
the indirect costs associated with distance education such as faculty development, support 
service budgets, partnership opportunities, and student retention (Clark et al., 2009). 

Analysis of the pillars. The Sloan-C Five Pillars of Quality Distance Learning provides 
institutions a guideline for the development of full programs to deliver online to students 
(Bourne et al., 2005; H. Wang, 2008). Through the five domains, institutions have a wide range 
of interpretative ability that enables them to determine their own level and attainment of distance 
education quality (H. Wang, 2008). The model loosely addresses the delivery of course 
materials in the learning effectiveness domain, but it is more focused on the construction and 
institutional support of the course (Bourne et al., 2005; Clark et al., 2009; H. Wang, 2008). 

While effective at the institutional assessment level of quality, it fails to adequately address uses 
of distance education technology, teaching practices, or learning activities in online courses 
(Bourne et al., 2005; Kee Meng & Mayadas, 2010; Westerfelt, 2011). As such, the institutions 
should develop an internal quality framework to support and evaluate course delivery in the 
online classroom (Bourne et al., 2005; Kee Meng & Mayadas, 2010; Westerfelt, 2011). 

Quality considerations in distance course delivery. The current and most widely used 
quality models for distance education, the Sloan-C Five Pillars of Quality in Distance Learning 
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and Quality Matters rubrics, address different levels of quality for distance education, but do not 
address the delivery of instruction in online courses (Battin-Little, 2009; Bourne et al., 2005; Kee 
Meng & Mayadas, 2010; Pollacia & McCallister, 2009; Westerfelt, 2011). Some scholars 
propose that faculty members teaching in distance courses do not have the initial or ongoing 
skills to deliver quality online courses due to the variety of techniques, distance learning 
technologies, and pedagogical practices possible in online coursework (Al-Salman, 2011; 
Graham & Jones, 2011; Hae-Deok et al., 2011; LaPrade et al., 2011; Lee et al., 2010; Singleton 
& Session, 2011; Tabata & Johnsrud, 2008). Distance education is an evolving modality of 
learning, and as such, many institutions struggle with adopting a macro-level model of quality 
because they do not address the delivery needs of their online courses (Daukilas et al., 2008; 
Sarrico et al., 2010) and thus have a need to create their own frameworks for the delivery of 
distance courses (Daukilas et al., 2008). 

Distance education faculty must choose the interactions, activities, and pedagogical 
practices they deem best suited for the course content and objectives (Tanner et al., 2009b). 
Students are not interested in traditional content-centric models of course design and delivery, 
but rather want to see robust and high quality engaging content, activities, interactions, and 
assessments (Ascough, 2011). The challenge is for colleges and universities to develop distance 
courses that capture what is good about face-to-face classes, leverage the right technology for the 
instructional goals for the course, and still allow for flexibility in the teaching and learning 
experience (Schulte, 2010). As such, institutions must invest resources into supporting faculty 
members in creating quality courses that properly use a variety of learner-centric items in 
distance courses (Schulte, 2010; Tanner et al., 2009b). The development of such models for 
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effective distance course delivery should be undertaken using a collaborative approach of faculty 
and instructional designers (H. Wang, 2008). 

Distance courses should address issues of course logistics, course design and 
organization, faculty-to-student interactions, student-to-student interactions, learning activities, 
student assessment, and course evaluation (Gaytan, 2009; Martinez-Argelles et al., 2010). As 
inclusions to an institutional model for quality course delivery it provides a substantial portion of 
the satisfaction and success of students (Martinez-Argelles et al., 2010). Faculty members are 
more inclined to instruct distance courses when quality course delivery and design are accounted 
for using a framework that supports effective teaching and learning (Tabata & Johnsrud, 2008). 

Course logistics. Class size or faculty-to-student ratio is an important aspect of the 
logistics and management of a distance course and contributes directly to the quality of the 
course (Orellana, 2006). In a 2009 published study of distance education administrators, 88% of 
respondents felt that online courses allowed the institution to enroll more students in a given 
section (Gaytan, 2009). An earlier 2004 published study about administrator concerns in 
distance education indicated that respondents did not feel distance courses had the potential for 
large enrollments (Chien et al., 2004). Gaytan’s (2009) report indicated that 38% of 
administrator respondents had knowledge that lower class sizes were more effective for student 
learning. 

Faculty members, in some cases, report that distance courses had enrollments exceeding 
100 students with an average of over 30 students per course section (Tanner et al., 2009b). A 
2006 published study regarding class sizes for online courses indicated a mean average of 22.8 
students with most course enrollments falling below 20 (Orellana, 2006). Distance education 
faculty reported that class sizes over 20 students jeopardized the levels of interaction, feedback. 
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and assessment that instructors can provide in an online class (Schulte, 2010). Schulte (2010) 
also reported that when class sizes exceeded 20 faculty were often reluctantly forced to take 
shortcuts or seek out alternative approaches to provide the minimally acceptable level of 
instruction to all students. 

Some research suggests that to have a highly engaging and interactive class less students 
per section is needed (Gaytan, 2009; Orellana, 2006). This links course interactivity to the effort 
and time of the faculty member responsible for the delivery of the course (Orellana, 2006). 
Orellana (2006) suggested that class sizes are best determined by comparing the number of 
students in a distance course to the types of interactions desired by the faculty member. These 
interactions include: instructional activities, social engagement, interactivity of technology, 
student participation and faculty involvement (Orellana, 2006). This suggests that class size for 
courses with lower desired interactivity could have higher enrollments and vice versa (Orellana, 
2006). 

Another important aspect of course logistics is the concept of learner responsibility in the 
online course (Daukilas et al., 2008; S. A. Lei & Gupta, 2010). Students taking distance courses 
need to exhibit a high degree of self-management and motivation to be successful (Daukilas et 
al., 2008; S. A. Lei & Gupta, 2010). In addition, students need to have a high degree of comfort 
or competence with using the technology required for the course (Daukilas et al., 2008). 

Research indicates that students do not fully understand the tacit skill development possible in 
course activities and assignments and as such it is critical to link these activities to both the 
responsibility of the student and potential learning outcomes intended in each activity (Ascough, 
2011). Statements of student responsibility in distance courses should include aspects of 
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readiness for independent learning, confidence with technology, comfort with using student 
support systems, and accessibility to the instructor (Conrad & Pedro, 2009). 

A similar facet of course logistics is the responsibility of faculty members (S. A. Lei & 
Gupta, 2010). Students taking online courses vary in their approach to learning from passive to 
active learners; faculty members should determine students’ perspectives of learning and align 
those to the course objectives (Kirkwood & Price, 2008). The nature of distance courses often 
means that faculty members find themselves as facilitators of knowledge, which requires 
establishing clear guidelines and expectations for both students and themselves (Mancuso, 2009). 
These guidelines should include a clear statement of the faculty member’s philosophy of 
teaching and learning and how that will manifest in the teaching of the course (Ascough, 2011). 
Finally, faculty members in distance courses should create more methodical and consistent 
approaches to the delivery of the course ensuring that can adequately interact and support 
students (Al-Salman, 2011; S. A. Lei & Gupta, 2010). 

The underlying structure of any distance course is the technological infrastructure that 
supports student learning and success (Daukilas et al., 2008). It is important to understand that 
computing technology should not dictate the pedagogy of the course (Kirkwood & Price, 2008; 
Schulte, 2010). Instead, technology should be selected when it meets the needs and expectations 
of the faculty, students, and course objectives (Schulte, 2010). When selected, the computing 
and distance education technologies should allow students and faculty members to effectively 
and easily collaborate, and engage the content of the course (Kurtz & Sponder, 2010). Use of 
technological tools in distance education varies by course and instructor, to this end, faculty 
member should be careful to outline course-specific use of any technology included in the course 
(Kirkwood & Price, 2008). Finally and importantly, faculty members should ensure that 
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computing technology works effectively to meet the needs of the course prior to the course 
beginning (Ward, Peters, & Shelley, 2010). 

A last important aspect of course logistics is the notion of accessibility to learning 
(Daukilas et al., 2008). Early distance courses did not consider the idea of students with 
disabilities or different learning needs and thus limited access to those that could participate with 
without accommodation (Engleman, 2005). Accessibility improvements were initially legally 
required but now accessible learning falls into the universal design mindset which makes 
learning available to a variety of students not merely those with accommodation needs 
(Engleman, 2005). When establishing courses faculty members should make students aware of 
any support services available at the institution, but also within the content by way of multiple 
forms of content delivery, activity participation, and assignment choice (Engleman, 2005). 

Course design. Course design concerns the instructional design of the distance course 
(Daukilas et al., 2008). Faculty members should carefully organize the course so it follows a 
logical, clear, and methodical format that is predictable and easy for students to navigate (S. A. 
Lei & Gupta, 2010). Students in distance courses indicated a high level of success and 
satisfaction when they could easily locate, use, and interact with the content of the course (S. A. 
Lei & Gupta, 2010). 

Learning objectives are an important element in any course design and should state 
clearly the knowledge, skills, behaviors, and attitudes students are expected to demonstrate upon 
successful completion of the course (Bers, 2008). Objectives should scaffold properly from 
lower order thinking skills to higher order thinking skills to help student effective complete the 
course (Ascough, 2011). Students express satisfaction with the course and outcomes when 
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faculty not only outline what is expected, but also what return on student investment is possible 
when completing each objective (Ascough, 2011). 

Student assessment in distance course is different than a face-to-face course (S. A. Lei & 
Gupta, 2010). Faculty members should consider more alternative or authentic style assessment 
activities that leverage the technology used in the online class (S. A. Lei & Gupta, 2010). 
Assessment devices should be clearly linked to course objectives to help students understand 
why the assessment is important and what they are expected to do as part of the activity 
(Ascough, 2011; Bers, 2008; S. A. Lei & Gupta, 2010). Faculty members should provide clear 
and detailed directions and requirements for assessments to aid students in understanding the 
expectations of each assignment (Ascough, 2011). Outputs for each assessment device should 
also be demonstrably clear to students and utilize as much objectivity as possible to avoid 
misunderstandings between the student and faculty member (Ascough, 2011). 

Instructional activities. Instructional activities form the largest portion of any distance 
course (Daukilas et al., 2008; S. A. Lei & Gupta, 2010). The area encompasses activities 
between faculty and students, for example, class synchronous and asynchronous discussions, 
content delivery, use of multimedia to inform students (Daukilas et al., 2008; S. A. Lei & Gupta, 
2010; Schulte, 2010; Stewart, Harlow, & DeBacco, 2011). The instructional activities area also 
includes interactions amongst students such as formal and informal discussions, collaborative 
projects, and social interactions (Daukilas et al., 2008; Kupczynski et al., 2012; S. A. Lei & 
Gupta, 2010; Riley & Anderson, 2006; Stewart et al., 2011; Ward et al., 2010; Zembylas, 2008). 
Learning activities are interactions in the class that promote the understanding of course 
materials and take many forms within the online class such as use of blogs, collaborative 
projects, presentations, journaling (Brodahl, Hadjerrouit, & Hansen, 2011; Chang, Morales- 



67 


Arroyo, Than, Tun, & Wang, 2010; Gaytan, 2009; S. A. Lei & Gupta, 2010). Finally, student 
assessment forms another part of the instructional activities area of online course delivery and 
takes the familiar form of examinations, papers, projects, and other assignments (Furnborough & 
Truman, 2009; Schulte, 2010; Ying, Huamao, Ronghuai, Yanhua, & Jingjing, 2008). 

Distance courses are sometimes perceived as isolated learning experiences for students 
(S. A. Lei & Gupta, 2010). The experience does not need to be a solo endeavor for students 
when faculty plan and execute deep and well-researched interaction activities between 
themselves and the students in the class (Gaytan, 2009). Interactions between students and 
faculty members should encourage students to evaluate and share their responses to the materials 
(Zembylas, 2008). Well executed faculty-to-student interactions can provide a deeper learning 
experience, promote higher order thinking skills, and create an class environment that motivates 
students to think critically (S. A. Lei & Gupta, 2010; Zembylas, 2008). As such, these types of 
instructional activities are an important element in both faculty and student satisfaction with 
distance education (Bolliger & Wasilik, 2009). Effective faculty-to-student interactions should 
be aligned to the course objectives, materials, and lesson goals (Al-Salman, 2011). Such 
interactions should also promote a sense of community and collegiality including the use of both 
synchronous and asynchronous tools like video, chat board, and discussion forums (S. A. Lei & 
Gupta, 2010; Schulte, 2010; Stewart et al., 2011). 

As Gaytan (2009) illustrated, distance learning is not an isolated or solo experience for 
students. In addition to faculty-to-student interactions, a faculty member should include student- 
to-student activities that engage students in collaborative dialogue and projects (Al-Salman, 
2011; Bolliger & Wasilik, 2009; Dixson, 2010; Gaytan, 2009). Student-to-student activities are 
ones that encourage students to work together on creating a deeper understanding of important 
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course topics (S. A. Lei & Gupta, 2010; Stewart et al., 2011). In many instances these 
interactions amount to facilitated discussion boards amongst the full class or learning groups 
(Kupczynski et al., 2012; Riley & Anderson, 2006). These types of inclusions into a course 
create the added benefit of students engaging in a social environment where they are more likely 
to explore their thoughts, emotions, and feelings about a particular topic (Kirkwood & Price, 
2008; Zembylas, 2008). 

Another facet of instructional activities in a distance education course is the concept of 
learning activities that engage students with an active task to deepen their levels of understanding 
about the course or lesson topics (Dixson, 2010; Gaytan, 2009). These activities should be 
aligned to course objectives and explained to students in a way that promotes their understanding 
of the goals of the course and their learning (Ascough, 2011; Gaytan, 2009). Research indicates 
that when included into distance courses student engagement and time on content mastery 
increases (Dixson, 2010). When implemented in the online class these learning activities often 
take the form of small projects, current event assignments, lab work, journaling, collaborative 
documents, and similar activities that support larger student assessment practices (Brodahl et al., 
2011; Chang et al., 2010; Dixson, 2010; S. A. Lei & Gupta, 2010). As such, these learning 
activities provide more opportunity for faculty to work with students and students to collaborate 
together promoting the other areas of this type of course inclusion (Dixson, 2010). 

Lastly, instructional activities include the use of student assessment in distance courses as 
a means to for both students to demonstrate understanding, but also for faculty to provide 
valuable feedback to students about competency and progress (Furnborough & Truman, 2009; 
Ifenthaler, 2010; Ying et al., 2008). Assessments often take the form of examinations, tests, or 
quizzes in an online course, but research does not clearly indicate whether or not students can 
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adequately demonstrate knowledge of course materials using online testing activities (Kirkwood 
& Price, 2008; Schulte, 2010). Faculty should plan and implement alternative and authentic 
types of assessment such as written assignments, study guides, projects, portfolios, and other 
devices that adequately align to course objectives and provide clarity into student understanding 
(Grabe & Flannery, 2010; Kirkwood & Price, 2008; Schulte, 2010). Students enrolled in 
distance classes require frequent and quality feedback on assessment activities and perceive a 
cause and effect relationship feedback has on their individual success in the course (Furnborough 
& Truman, 2009). Feedback on assessment also encourages students to confidently engage with 
faculty and classmates promoting a social and collaborative learning environment (Ifenthaler, 
2010; Ying et al., 2008). 

Gap in Research and Literature 

Some scholars advised that assessing and implementing academic quality for distance 
education courses is limited by existing quality models that do not adequately address the 
delivery of instruction in online classes (Battin-Little, 2009; Bourne et al., 2005; Endean et al., 
2010; Forsyth et al., 2010; Kee Meng & Mayadas, 2010; Picciano, 2009; Pollacia & McCallister, 
2009; Postek et al., 2010; P. S. Smith, 2011; Westerfelt, 2011). The literature reveals that many 
faculty members teaching in distance education courses lack pedagogical and technological skills 
to deliver effective instruction due to the numerous and evolving techniques, tools, and practices 
possible in online courses (Al-Salman, 2011; Daukilas et al., 2008; Graham & Jones, 2011; Hae- 
Deok et al., 2011; Kupczynski et al., 2012; LaPrade et al., 2011; Lee et al., 2010; Orr et al., 

2009; Sarrico et al., 2010; Singleton & Session, 2011; Tabata & Johnsrud, 2008). This breach in 
academic quality processes poses risks for institutions of higher education due to the increasing 
external and internal demands to deliver and report the quality of distance learning programs and 
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courses (Daniel et al., 2006; Gaytan, 2009; Jung et al., 2011; Peinovich, 2008; Sener, 2010; 

Seok, 2007). Some scholars suggest that addressing this breach in distance education quality 
processes requires the development of institutionally specific systems and processes by which 
faculty members are supported in selection of pedagogical activities that provide the freedom 
and flexibility to meet course objectives using well-research and evidence-based techniques (Al- 
Salman, 2011; Forsyth et al., 2010; Graham & Jones, 2011; Hae-Deok et al., 201 1; Huett et al., 
2008; Kupczynski et al., 2012; Lee et al., 2010; Orellana, 2006; Orr et al., 2009; Postek et al., 
2010; Schuck et al., 2008; Singleton & Session, 2011; Tabata & Johnsrud, 2008). 

Summary 

Chapter 2 provided a literature review of the topics: quality in higher education and 
quality in distance education with topical areas pertinent to the present Formative Research study 
of developing a quality framework for delivery of distance education courses at a small liberal 
arts institution in western New York state. Quality in higher education is a broad subject with 
many facets and topical areas to be covered, it is suggested that regardless of macro-level models 
or major philosophies that institutions design their own approach to quality using a variety of 
approaches and factors (Harvey & Green, 1993; Pratasavitskaya & Stensaker, 2010; Singh, 2010; 
Tanner et al., 2009b). Narrowing the topic of quality in higher education to the modality of 
distance learning, there is equal pressure on institutions to provide evidence of quality instruction 
in distance learning courses (Daniel et al., 2006; Jung et al., 2011; Peinovich, 2008; Seok, 2007). 
Like the broader quality in higher education issue, macro models for quality in distance learning 
do not provide an adequate model for portraying the delivery of instruction at the course level 
(Daniel et al., 2006; Gaytan, 2009; Jung et al., 2011; Peinovich, 2008; Sener, 2010; Seok, 2007). 
To this end, institutions are encouraged by some scholars to develop a quality tool that provides 
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faculty members guidance, support, training, and quality guidelines specific to course level 
instructional delivery as part of the overall quality program (Al-Salman, 2011; Forsyth et al., 
2010; Graham & Jones, 2011; Hae-Deok et al., 2011; Huett et al., 2008; Kupczynski et al., 2012; 
Lee et al., 2010; Orellana, 2006; Orr et al., 2009; Postek et al., 2010; Schuck et al., 2008; 
Singleton & Session, 2011; Tabata & Johnsrud, 2008). 

The purpose of the present Formative Research study was to develop an instructional 
delivery framework for distance education courses at a small private liberal arts university in 
western New York state. The framework must meet requirements from accreditation and 
regulatory agencies, faculty and administration concerns about how to design and implement 
quality distance courses, and maintenance of pedagogical flexibility for faculty. Instructional 
delivery items should be well-researched and evidence-based to provide adequate guidance to 
experienced and inexperienced distance education faculty. Data were collected using 
observations of study participants, semi-structured interviews, and focus groups. This chapter 
provided a review of literature on topics relating to higher education quality, distance education 
quality, and course design, and course delivery and the content of the research questions evolved 
from the discussions. The following chapter contains information regarding the present study at 
Magdalene University including elements of research design and research methodology. 
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Chapter 3: Research Method 

Scholars concerned with quality in distance education demonstrate a gap in current 
practice and empirical research by advocating for institutional-specific and delivery focused 
quality frameworks to be constructed for support of distance education faculty (Al-Salman, 
2011; Forsyth et al., 2010; Graham & Jones, 2011; Hae-Deok et al., 2011; Huett et al., 2008; 
Kupczynski et al., 2012; Lee et al., 2010; Orellana, 2006; Orr et al., 2009; Postek et al., 2010; 
Schuck et al., 2008; Singleton & Session, 2011; Tabata & Johnsrud, 2008). The purpose of this 
Formative Research study was to develop an instructional delivery framework for distance 
education courses at a small private liberal arts university in western New York state referred to 
with the pseudonym Magdalene University. The framework meets requirements from 
accreditation and regulatory agencies, faculty, and administration concerns about how to design 
and implement quality distance courses, and maintenance of pedagogical flexibility for faculty. 
Instructional delivery items were well-researched, evidence-based, and institutionally sound to 
provide adequate guidance to experienced and inexperienced distance education faculty. The 
Data were collected using observations of study participants, semi-structured interviews, and 
focus groups. 

Chapter 3 provides the research methodology to be used in responding to the research 
questions. Topics treated in this chapter include: research design and appropriateness, research 
questions, population, geographical location, informed consent, role of the researcher, data 
collection, issues of validity and reliability 

Research Design and Appropriateness 

Formative Research is a developmental methodology that focuses on explaining to 
educators how to design instruction not on defining instructional design which is integral to an 
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applied field like education (Reigeluth & Frick, 1999). The method draws from an instructional 
design process known as formative evaluation and case study methods with focus on an 
instructional design process (Reigeluth & Frick, 1999). The methodology is used for two major 
categories of instructional design research: designed case studies and naturalistic studies 
(Reigeluth & Frick, 1999). Designed case studies are used to develop a new or improve an 
existing instructional design theory, model, or system (Reigeluth & Frick, 1999). Naturalistic 
studies focus on the creation or improvement of instructional design systems not ascribed to a 
particular theory but serving the same goals as a designed study (Reigeluth & Frick, 1999). 
Corresponding to the purpose of the study, the present research followed the Designed Case 
Formative Research approach to design a new instructional delivery framework appropriate for 
quality online course development. The Designed Case Formative Research follows a five-step 
process that contains an iterative cycle at step four: 

1 . Create a case in supporting the new instructional design framework. 

2. Collect and analyze formative data. 

3. Revise the instructional design framework. 

4. Repeat steps two and three until the research reaches a point of saturation. 

5. Fully develop the tentative instructional design framework. (Reigeluth & Frick, 
1999; Reigeluth, 2009) 

This differs from a Designed Case that seeks to improve an existing theory or model where the 
process begins with the selection of a that theory or model (Reigeluth, 2009). The following 
process was followed in the present study. 

Step One: Create a case for supporting the new instructional design framework. In 

step one, a situation is selected that best exemplifies common situations in which the new model 
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was applied, then develops a case that applies to the situation (Reigeluth & Frick, 1999). It is 
common for the case be a product, implementation of a product, process, or a combination 
thereof (Reigeluth & Frick, 1999). The case is developed from experience, intuition, knowledge, 
and education of investigator or research team (Reigeluth & Frick, 1999). The research provides 
guidelines for the selection and use of the individual elements to support the case becoming an 
instance of the model being developed (Reigeluth & Frick, 1999). 

Step Two: Collect and analyze formative data. With a case developed that provides 
study participants with the present instructional delivery process, formative evaluation of the 
case and model begins. Formative evaluation data is collected from study participants through 
observations of their interactions with the present model, reviews their produced work based on 
the model, conducts singular interviews with participants, and engages focus groups in 
discussions. Reigeluth and Frick (1999) contended that the use of observation, document review, 
and interviews are the most useful data collection techniques when performing a Formative 
Research Case Study. Interviews and focus groups take place as soon after study participants 
have applied the model to increase external validity of the research (Reigeluth & Frick, 1999). 

Step Three: Revise the instructional design framework. The model is revised based 
on formative evaluation data gathered from the study participants (Reigeluth & Frick, 1999). 

This process can begin at any point during Step Two when it is determined that the information 
provide has value (Reigeluth & Frick, 1999). A detailed change log is kept to track both the 
revisions to the model as well as future studies (Reigeluth & Frick, 1999). 

Step Four: Repeat steps two and three until saturation. Collection of formative 
evaluation data and revisions of the framework repeat until no further substantive revisions are 
provided by the participants (Reigeluth & Frick, 1999). To the extent possible participants’ 
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academic disciplines are identified (Reigeluth & Frick, 1999). These variances are intended to 
increase generalizability and identify anomalies where some aspects of the framework work 
differently for different situations (Reigeluth & Frick, 1999). 

Step Five: Fully develop the tentative instructional design framework. Upon 
reaching a point of saturation, final revisions to the design framework are completed and a 
tentative framework is published (Reigeluth & Frick, 1999). Any inclusive weaknesses, 
anomalies, or inadequacies are published with the model (Reigeluth & Frick, 1999). Finally, the 
published framework corresponds with suggestions for future studies that could aid in continued 
improvement of the framework (Reigeluth & Frick, 1999). 

Appropriateness of the method. Qualitative research crosses disciplinary boundaries 
and enables the exploration of the interconnectedness and complexities of issues without 
narrowly attempting to prove or disprove a set of variables (Denzin & Lincoln, 2011; Lichtman, 
2013). In the field of education, use of qualitative research is appropriate because many aspects 
of the field use non-numerical data largely in the form of words or visual images whereas any 
quantitative research would require the use of numerical data (Lichtman, 2013). Where 
quantitative research methodologies seek to test hypotheses and look for cause and effect as 
related to several variables, qualitative studies seek to understand and interpret whole social 
phenomena (Denzin & Lincoln, 2011; Lichtman, 2013). Quantitative research is difficult in the 
field of higher education because the variables that erode reliability and validity are difficult to 
isolate in such a social environment (Van de Vord & Pogue, 2012). Finally, qualitative 
methodologies use inductive reasoning by beginning with the observations of specific items and 
using that data along with researcher interpretations to provide an understanding of some larger 


concept (Lichtman, 2013). 
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Formative Research, as a qualitative research method, is derived from the case study 
methodology. Specifically, Formative Research follows the exploratory holistic single case 
procedures (Reigeluth & Frick, 1999; Reigeluth, 2009). Single holistic case studies or Type 1 
case studies are appropriate for use when no clear sub-unit of the case exists and when the 
underlying theory of the case is holistic in nature (Yin, 2009). Formative Research follows Yin’s 
guidelines for appropriate use of Type 1 studies because the research is focused on the 
development of one instructional design theory, model, or framework (Reigeluth & Frick, 1999). 

Yin (2009) cautioned that single-holistic case studies can change or shift during research, 
creating problems for investigation and interpretation. Formative Research includes an iterative 
cycle that requires to continuing of data collection, analysis, and revisions until the study reaches 
a reasonable level of saturation (Reigeluth & Frick, 1999). This iterative cycle addresses any 
shifting or evolving issues that might cause interpretation difficulties throughout the 
investigation (Reigeluth & Frick, 1999). Another concern with single holistic case studies is that 
these studies can unintentionally remain abstract and theoretical (Yin, 2009). Since Formative 
Research is intended to apply instructional design theories or models of practice and action, the 
danger of being abstract is diminished by the process (Reigeluth & Frick, 1999). Yin (2009) 
explained that some researchers criticize case study methodology for lacking sufficient rigor. 

This criticism is addressed in Formative Research when issues of construct validity, rigorous 
data collection, sound analysis processes, and a focus on generalizability are addressed 
(Reigeluth & Frick, 1999). Single institution case studies regarding quality in higher education 
offer insight and knowledge about institutional approaches and can be relevant to other similar 
colleges or universities (Pratasavitskaya & Stensaker, 2010). 
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Formative Research requires that the principle investigator and study participants engage 
in reflective thinking about their instructional practices and courses to find improvements 
throughout the research process, particularly in the iterative collection, analysis, and revision 
phases (Reigeluth & Frick, 1999). This reflective thinking aspect of the research design mirrors 
calls for reflective thinking about higher education quality. Quality processes in post-secondary 
education should include reflective thinking about instruction by an engaged group of faculty 
(Daukilas et al., 2008; Jordens & Zepke, 2009; Oermann, 2007; Pratasavitskaya & Stensaker, 
2010). This processes supports the constructivism where reality is a product of interpretations 
and contributions of the individuals involved, which parallels what is called for in the Formative 
Research study method (Juceviciene, 2009; Reigeluth & Frick, 1999). 

Reflective thinking of participants was captured through audio recording and accurate 
transcription of observations, semi-structured interviews, and focus groups. These reflections 
were verified through member checks of transcriptions made available to each participant. 

Audio recording and noting reflective thinking aided in developing rationale for making changes 
to the design theory (Reigeluth & Frick, 1999). The capture of my reflective thinking was 
accounted by noting the nature of each revision made (Reigeluth & Frick, 1999). 

Appropriateness of the design. The Formative Research study method engages a group 
of practioners to find and implement improvements in instructional design models (Reigeluth & 
Frick, 1999). This method is appropriate to the problem of supporting quality in higher 
education. Curriculum quality is enhanced when a group of concerned practioners come together 
in a community of practice around issues of course quality (Daukilas et al., 2008; Jordens & 
Zepke, 2009). Because course quality is neither objective nor standard, it requires the 
collaboration of educational communities to provide their understanding and clarity around 
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academic quality issues like course design and delivery (Jordens & Zepke, 2009). Distance 
education administrators and leadership promote a collaborative approach when working with 
issues of course design and implementation (Seok, 2007). Such project studies benefit from 
utilizing faculty in planning, developing, implementing, and ongoing evaluation of distance 
courses (Gaytan, 2009; Jordens & Zepke, 2009). 

Formative Research is an established research method and used in a variety of studies 
relating to the present research. As a method to investigate systemic and evaluation models 
Formative Research was used by Doblar (2010) and Schankman (2006). In the study of 
incorporating technology into instruction the method was used by Enfield (2012), Hsu (2009), 
Roskos, et. al. (2011), Squire (2008), and F. Wang and Hannafin, (2005). Specifically, the 
Formative Research method was used for distance education design studies by Halverson (2006), 
Snyder (2006), and Yagodzinski (2012). 

Research Questions 

Research questions focus the study and guide the research as it progresses (Denzin & 
Lincoln, 2011). The goals of the Formative Research study with the purpose of designing a new 
instructional design frameworks are to meet the demands for appeal, ease of use or effectiveness, 
and efficiency or ease of use (Reigeluth & Frick, 1999). The Formative Research study explored 
the following research questions: 

RQ1: What are the perceptions and attitudes of faculty in a small university about how a 
proposed distance education quality course delivery framework is appropriate and appealing to 


them? 
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RQ2: What are the perceptions and attitudes of faculty in a small university about how a 
proposed distance education quality course delivery framework will meet their needs for 
effectiveness in delivering quality online courses? 

RQ3: What are the perceptions and attitudes of faculty in a small university about how a 
proposed framework will meet their needs for ease of use? 

Population 

The study participants were faculty members of Magdalene University. Potential 
participants were invited to participate by electronic mail invitation sent from the administrative 
director at Magdalene University (Appendix B). An electronic mail reminder was sent ten days 
after the original electronic mail invitation (Appendix B). Volunteers were asked to contact the 
principle investigator directly to volunteer as participants for the research. Study participants 
came from a wide range of academic disciplines including arts, sciences, health, and human 
service related departments. Study participants were both tenured and non-tenured faculty 
currently teaching at the research site institution. Study participants had a variety of experiences 
teaching distance education courses ranging from no familiarity to substantial proficiency with 
this modality. The study concluded with ten faculty members included in the research study. 
Formative Research requires the investigator to continue iterative cycles of data collection and 
revision until participants offer no substantive revisions to the instructional delivery framework 
(Reigeluth & Frick, 1999). 

Informed Consent 

A letter was sent prior to initiation of the research proposal to the senior academic 
administrative leader at the Magdalene University for permission to use the location and faculty 
for the study, which was returned with an affirmative answer. The letter included a statement of 
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the study purpose, expectations, and assurances of confidentiality and legitimacy of the study. 
Upon confirmation of the research proposal by the Institutional Review Boards at the university 
that is the target of the study and the University of Phoenix pursuant to the U.S. Federal 
Government Department of Health and Human Services (2009) regulation 45 CFR § 46.10, the 
Formative Research study commenced. 45 CFR § 46.10 states the probability and magnitude of 
harm or discomfort anticipated in the research should not be greater in and of themselves than 
any ordinarily encountered in daily life, or during the performance of routine physical or 
psychological examinations or tests, and the results are that the study was deemed to be one of 
minimal risk to participants pursuant to the law. 

A list of the Magdalene University electronic mail addresses of all faulty was obtained. 
Potential participants were invited to participate by electronic mail invitation sent from an 
administrative director at Magdalene University (Appendix B). A subsequent electronic mail 
reminder was sent ten days after the original electronic mail invitation (Appendix B). Volunteers 
were asked to contact me directly to volunteer as participants for the research. The invitation 
contained a clear statement of the research study, expectations of the study participants, 
assurances of confidentiality and privacy protections, statement of risks and benefits, procedure 
for how to discontinue as a study participant, and request to contact me directly to participate. 
Once study participants agreed to take part in the research study they were provided an informed 
consent form (Appendix C) that outlined all the information contained in the recruitment email. 
Interviews and focus group sessions were audio-recorded. 

A purposive selection of participants was made from those who responded positively to 
the letter of invitations to ensure that a cross section of disciplines and faculty were included in 
the interviews and focus groups. A time and place convenient to both parties was selected and 
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individual interviews were no longer than 1 hour in length. Each participant participated in 
multiple forms of data collection as noted in a following paragraph. An Interview Protocol 
(Appendix D) was used to ensure consistency across all interviews. Interviewees were asked a 
group of 13 questions related to the three research questions. The questions were purposively 
constructed from Formative Research studies and derived from Reigeluth and Frick’s (1999) 
seminal guidance on this research method. The questions were designed and organized to 
address the three areas of Formative Research: appeal, effectiveness, and efficiency or ease of 
use (Reigeluth & Frick, 1999). Interviewees were allowed to diverge from the interview 
questions as desired. The interviewer recorded observational notes. A transcription of the 
recordings of individual interviews was implemented, and interviewees were provided with an 
opportunity to review the transcriptions for accuracy. 

A Focus Group Protocol (Appendix E) was used to ensure research objectives are met. 
The questions were purposively constructed from Formative Research studies and derived from 
Reigeluth and Frick’s (1999) seminal guidance on this research method and the Interview 
Protocol from this study (Appendix D). The questions were designed and organized to address 
the three areas of Formative Research: appeal, effectiveness, and efficiency or ease of use 
(Reigeluth & Frick, 1999). Participants were asked six questions to prompt discussion. The 
focus group participants were allowed to deviate from the questions. An audio recording was 
taken of each focus group proceeding. A transcription of the recordings of focus group was 
implemented, and participants were provided with an opportunity to review the transcriptions for 
accuracy. 

Only I have access to the interview and focus group data. Transcripts of data were coded 
(example: 1001 for first interviewee). Audio recordings were deleted from the password 
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protected computer and device once transcriptions were complete. A master code list connecting 
names of subjects to code numbers is stored in a locked cabinet in principle investigator’s office 
and will be destroyed in 3 years after publication of the dissertation. Upon the request of a 
participant, the principle investigator who has sole and full access to the secured and protected 
research documents, will remove and shred all data related to any participant including focus 
group data that specifically identifies or originates from the participant. All data pertinent to the 
study will remain in a safe location for 3 years after publication of the dissertation, after which is 
will be destroyed. 

Geographic Location 

The research took place, with permission, at a western New York state private, not-for- 
profit four-year university. The institution provides degree programs at the Bachelors, Masters, 
and clinical doctorate levels in areas of humanities, sciences, and health sciences. The university 
operates on a semester system with a January session and two intersessions. The influencing 
criteria for selecting Magdalene University as the research site was the location of the school. 

Within western New York state, the institution lies in Erie County. As of 2011, Erie 
County has a population 918,028; of which 81.1% are White, 13.9% are Black, 4.7% are 
Hispanic, and approximately 5% are of other decent (“Erie County QuickFacts from the US 
Census Bureau,” n.d.). The population is comprised of 51.7% female and 48.3% male persons 
(“Erie County QuickFacts from the US Census Bureau,” n.d.). Educationally, 88.9% of the Erie 
County population over the age of 25 have a high school of higher degree and 29.8% of the 
population of Erie County 25 years or older possess a Bachelor degree or higher (“Erie County 
QuickFacts from the US Census Bureau,” n.d.). The research site institution has an immediate 
geographic enrollment source from 96 public school districts with a total of 117 public high 
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schools (“WNYRIC School Districts □: Western New York Regional Information Center 
(WNYRIC)D: Erie 1 BOCES,” n.d.). 

Data Collection and Analysis 

Data collection in Formative Research is influenced by thoroughness and credibility 
(Reigeluth & Frick, 1999). Thoroughness of data collection is enhanced when data collection 
procedures include advanced preparation of the study participants, emersion in data collection 
process with gradually decreasing obtrusiveness, and iterative phases of data collection and 
analysis until the study reaches saturation (Reigeluth & Frick, 1999). Credibility or reliability of 
data collection is established using triangulation of data collected, rigorous chains of evidence 
collection, and member checks (Reigeluth & Frick, 1999). 

Data were accurately transcribed and analyzed using triangulation techniques that utilize 
multiple study participants in iterative rounds of collection, analysis, and revision (Reigeluth & 
Frick, 1999). Each study participant took part in multiple forms of data collection: observation 
of study participants applying the instructional design framework, individual interviews 
conducted either face-to-face or via web conference technologies, and focus group interviews. 

Yin (2009) suggested that data collection should follow a strict chain of evidence that is 
clear and precise. Focus study participants gathered in a private location where their interactions 
with the instructional design framework were observed. Participants were individually 
interviewed in a private location that disallows outside interference or distraction; interviews 
were recorded digitally and transcribed into a database, the audio file deleted from the recording 
device once transcription is complete and confirmed. Focus group Data were recorded via 
digital voice recorder; Data were transcribed accordingly into the evidence database and the 
audio file deleted from the original sources. 
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Data were analyzed for concepts or themes and confirmation throughout the iterative 
cycles of data collection, analysis, and revision until the study reaches a point of saturation. 
Iterative cycles utilized different participants to increase the generalizability of the study 
(Reigeluth & Frick, 1999). Any differences that occurred in the iterations were noted and 
attached to some measure of specific situation and participant experience then accounted for in 
the framework to ensure more generalizability and usability by a variety of practioners 
(Reigeluth & Frick, 1999). Data were also be analyzed for duplicated evidence to support 
elements and revisions to the design framework (Reigeluth & Frick, 1999). 

Role of the Researcher 

The role of the researcher in qualitative research is a critical element to the study 
(Lichtman, 2013). Interpretations in the study were based upon my experience, education, and 
background (Lichtman, 2013; Reigeluth & Frick, 1999). As an administrator at Magdalene 
University, there is frequent contact with potential study participants and administrators. Daily 
responsibilities include issues of course development, instructional design, teaching and learning, 
and educational technology including online courses. 

Validity and Reliability 

This Formative Research study focused on the design of a new instructional design 
framework for delivering quality distance courses. Qualitative research that investigates the 
description of a situation or phenomena typically addresses concerns of validity and reliability 
(Reigeluth & Frick, 1999). Since Formative Research focuses on the design of a theory, model, 
or framework validity takes the form of preferability (Reigeluth & Frick, 1999). Likewise, 
reliability takes the form of generalizability (Reigeluth & Frick, 1999). 
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Validity when viewed through the lens of preferability focuses on the users’ experiences 
with the instructional design framework (Reigeluth & Frick, 1999). Preferability is generally 
ascribed to issues of effectiveness, efficiency, and appeal (Reigeluth & Frick, 1999). 
Effectiveness is determined the extent to which the design framework met the intended goal and 
usually measured by a numerical scale (Reigeluth & Frick, 1999). Efficiency is measured by 
considering the effectiveness of the design framework compared to the time involved for the 
users of the framework (Reigeluth & Frick, 1999). Appeal refers to the enjoyment users of the 
design framework express when applying the framework to their work (Reigeluth & Frick, 
1999). The research included interview questions that uncover participants’ feelings about 
effectiveness, efficiency, and appeal through a semi-structured interview protocol that asked 
about positive and negative aspects about using the framework to design a course. 

Reliability in Formative Research is expressed in terms of generalizability (Reigeluth & 
Frick, 1999). Reliability was determined by using research and analysis methods that increase 
the generalizability of the study as stated in the Data Analysis section above. The study uses 
triangulation of data, rigorous chains of evidence, and member checks. The study uses iterative 
cycles of data collection, analysis, and revision using varying groups of study participants with 
different situational or disciplinary concerns to increase the generalizability of the framework 
across interdisciplinary application. 

Summary 

This Formative Research study followed a qualitative case study methodology with a 
focus on creating a new instructional design framework (Reigeluth & Frick, 1999). The study 
designed a quality framework for use in determining what to include and how to utilize 
instructional activities in a distance education course. The study took place at Magdalene 



86 


University in western New York state and was comprised of faculty members from that 
institution. Chapter 3 provided the research methodology used in responding to the research 
questions. Topics treated in this chapter included: research design and appropriateness, research 
questions, population, geographical location, informed consent, role of the researcher, data 
collection, and issues of validity and reliability. 
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Chapter 4: Analysis and Results 

The puipose of this Formative Research study was to develop an instructional delivery 
framework for distance education courses at a small private not-for-profit university in western 
New York state referred to with the pseudonym Magdalene University in this study. The 
framework meets requirements from accreditation and regulatory agencies, faculty and 
administration concerns about how to design and implement quality distance courses, and 
maintenance of pedagogical flexibility for faculty. Instructional delivery items were well- 
researched, evidence-based, and institutionally sound to provide adequate guidance to 
experienced and inexperienced distance education faculty. 

Chapter 4 presents the analysis and results of the research creating the instructional 
delivery framework. The chapter provides demographic information about the ten participants in 
the study, data collection procedure, restatement of the research questions, and analysis of the 
data collected. Presentation of data analysis uses a narrative format derived from the described 
procedure for coding, categorizing, and development of concepts of themes. 

Participant Demographics 

The ten participants selected were a purposeful sample. Purposeful sample was selected 
to better understand the perceptions and attitudes college faculty regarding the appeal, 
effectiveness, and efficiency of the instructional delivery framework. Potential participants were 
invited to volunteer for the study by electronic mail invitation sent from an administrative 
director at Magdalene University (Appendix B). A subsequent electronic mail reminder was sent 
ten days after the original electronic mail invitation (Appendix B). The invitation contained a 
clear statement of the research study, expectations of the study participants, assurances of 
confidentiality and privacy protections, statement of risks and benefits, procedure for how to 
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discontinue as a study participant, and request to contact me directly to participate. The 
invitations was sent to 119 full time professors and 149 part time instructors (Institute of 
Educational Sciences, N.D.). Thirteen members of the faculty agreed to participate in the study 
and three were excused from the study once saturation of the feedback was obtained. Formative 
Research determines saturation when no substantive changes are requested or suggested by study 
participants (Reigeluth & Frick, 1999). Once study participants agreed to take part in the 
research study they were provided an informed consent form (Appendix C) that outlined all the 
information contained in the recruitment email. Interviews and focus group sessions were audio- 
recorded. 

Only some demographic information was collected about the participants to inform the 
purposeful selection of sample sets and avoid redundant departmental representation in the 
sample sets. This information included department, experience with distance courses, and 
employment status. The study population included seven full time and three part time faculty 
members. Amongst the population three members served as faculty in the Division of Health 
and Human Services while seven served in the Division of Arts and Sciences. The academic 
disciplines represented in the study population were Athletic Training, Education, English, 
Healthcare Studies, Anthropology, Mathematics, Modern Language, Natural Sciences, and 
Psychology. Involvement with distance education was unnecessary for participation in the 
research and participants casually remarked about their teaching experience with distance 
education technology. Five of the study participants had no experience with teaching distance 
courses, one had taught in only a hybrid or blended class modality, and four had experience 
teaching fully online courses. Table 1 provides a demographic explanation of the participants in 
the study. 
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Table 1 


Participant Demographics 


Study Group 

Participant 

Academic 

Discipline 

Distance 

Education 

Experience 

Group 1 

1001 

Education 

Fully Online 


1002 

Modern Language 

Fully Online 


1003 

Athletic Training 

No Experience 


1004 

Psychology 

No Experience 

Group 2 

2001 

Mathematics 

No Experience 


2002 

Modern Language 

Hybrid 


2003 

Healthcare Studies 

Fully Online 


2004 

English 

No Experience 

Group 3 

3001 

Natural Sciences 

Fully Online 


3002 

Anthropology 

No Experience 


Data Collection Procedure 

Data collection strictly followed the data research protocol described in Chapter 3: 
Research Methods. Once approval from the Institutional Review Boards was obtained an 
invitation was sent by electronic mail invitation from an administrative director at Magdalene 
University (Appendix B). A subsequent electronic mail reminder was sent ten days after the 
original electronic mail invitation (Appendix B). Volunteer participants were organized into 
purposeful sample sets of no more than four members each. Three sample sets were used for this 
study. Sample sets were constructed based on order of volunteer inquires into the study and 
participant availability with consideration of disciplinary focus to avoid duplicate department 
representation in any sample set. Participants in the study were informed that the goal of the 
research was to develop a new instructional delivery framework for distance education courses. 
The participants were instructed to be critical of the instructional delivery framework and to look 
for weaknesses in the model through each of the data collection steps (Reigeluth & Frick, 1999). 
Data were collected through observations, individual interviews, and focus group that included 


questions about direct use, perceptions, and attitudes of the participants regarding the use of the 
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instructional design framework provided. All study participants took part in all three methods of 
data collection. Ten Magdalene University faculty members participated in the study with 
varying employment statuses, disciplinary foci, and experience with distance education. 

Qualitative research that investigates the description of a situation or phenomena 
typically addresses concerns of validity and reliability (Reigeluth & Frick, 1999). Since 
Formative Research focuses on the design of a theory, model, or framework validity takes the 
form of preferability whereby the research procedures determine the desirability in terms of 
effectiveness, efficiency, and appeal of the proposed framework (Reigeluth & Frick, 1999). Data 
collection included interview and focus group questions that uncover participants’ feelings about 
effectiveness, efficiency, and appeal through a semi-structured interview and focus group 
protocols (Appendices D and E) that asked about positive and negative aspects about using the 
framework to design a course. 

Reliability in Formative Research is expressed in terms of generalizability and was 
determined by using triangulation of data collected between participants and in reference to 
literature, rigorous chains of evidence collection, and member checks of transcribed data 
(Reigeluth & Frick, 1999). The study uses iterative cycles of data collection, analysis, and 
revision using varying groups of study participants with different situational or disciplinary 
concerns to increase the generalizability of the framework across interdisciplinary application. 
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Research Questions 

Research questions focus the study and guide the research as it progresses. The goals of 
the Formative Research study with the purpose of designing a new instructional design 
frameworks are to meet the demands for appeal, effectiveness, and ease of use or efficiency 
(Reigeluth & Frick, 1999). The Formative Research study explored the following research 
questions: 

RQ1: What are the perceptions and attitudes of faculty in a small university about how a 
proposed distance education quality course delivery framework is appropriate and appealing to 
them? 

RQ2: What are the perceptions and attitudes of faculty in a small university about how a 
proposed distance education quality course delivery framework will meet their needs for 
effectiveness in delivering quality online courses? 

RQ3: What are the perceptions and attitudes of faculty in a small university about how a 
proposed framework will meet their needs for ease of use? 

Analysis and Results of the Study 

This section provides a qualitative data analysis regarding the implementation of a new 
instructional delivery framework intended to assist faculty in developing and implementing 
quality distance education courses. Within this section the data analysis procedure, research 
question analysis, and final instructional delivery framework analysis are treated. 

Data analysis procedure. Formative Research studies use data analysis to make 
revisions to and understand the impact of an instructional model, theory, or framework 
(Reigeluth & Frick, 1999). Data analysis was conducted for each of the three research questions. 
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Each research question analysis followed Lichtman’s (2013) Three Cs model where the 
transcribed data were analyzed using the following steps: 

1 . Initial coding that summarizes data. 

2. Revisiting initial coding to eliminate redundancies and consolidate codes 

3. Initial development of categories for finalized codes 

4. Modifying categories based on additional reading of data 

5. Revisiting categories to eliminate redundancies 

6. Moving from categories to concepts (themes). 

During the analysis, the transcribed data were annotated using codes befitting the content 
and purpose of the study. Coded data were revisited and new codes were ascribed to eliminate 
redundancies and consolidate data into categories. A document for each category was created 
and the appropriate coded data were copied from the transcriptions into those documents. 
Categorized data were read and reflected upon for critical meaning that formed the concepts or 
themes used to address each research question (Lichtman, 2013). The concepts for the analysis 
are pedagogical flexibility, teaching and learning quality, support and necessary skills, 
framework organization, and comprehensiveness. 

Conceptual understanding: Pedagogical Flexibility. Coded responses and categorical 
grouping indicated that being able to flex pedagogical practices was important to the participants. 
To promote instructional quality in distance learning institutions should not constrain or limit 
faculty with regard to instructional methods, tools, or techniques. Instructors require the ability 
to freely select and implement instruction as they deem appropriate to their abilities and course 
goals (Kirkwood & Price, 2008). Scholars support this pedagogical flexibility concept by 
suggesting that faculty members are better equipped to make pedagogical decisions for their 
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courses and students (Mancuso, 2009; McLawhon & Cutright, 2012; Selwyn, 2011). The 
decisions faculty members make range from agreeing and disagreeing with teaching practices to 
selection of practices based on course objectives and needs (Ching- San et al., 2009; Mustafa & 
Dalen, 2006; Yair, 2008). 

Conceptual understanding: Teaching and Learning Quality. Quality in distance 
education is achieved when pedagogy, not technology, comes first in the instruction (Kupczynski 
et al., 2012; Sarrico et al., 2010). Systems that focus on pedagogy and incorporate technology as 
means to achieve teaching and learning goals contribute to a higher degree of quality and faculty 
willingness to engage in distance education (Graham & Jones, 2011). The creation of such a 
system or framework should not rigidly prescribe teaching methods that meet quality 
expectations to avoid low faculty adoption of distance education (Kirkwood & Price, 2008; S. A. 
Lei & Gupta, 2010). A model or framework that allows faculty the flexibility to determine the 
best means to meet quality expectations may also encourage faculty to apply it to distance 
courses (Mancuso, 2009; Schuck et al., 2008; Yair, 2008). These flexible models work most 
effectively when faculty are able to determine how to link learning objectives, instructional 
methods, and student assessment (Bers, 2008; Ziliukas & Katiliute, 2008). 

Conceptual understanding: Support and Necessary Skills. Scholars suggest that 
faculty resistance to participating in distance education is due to a lack of support and the 
necessary skills to deliver a course (Al-Salman, 2011; Graham & Jones, 2011; LaPrade et al., 
2011; Lee et al., 2010; Singleton & Session, 2011; Tabata & Johnsrud, 2008). Faculty who are 
more familiar with tools and techniques for teaching at distance report being more willing to 
teaching online courses (Singleton & Session, 2011; Tabata & Johnsrud, 2008). Support systems 
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that create faculty the awareness are more likely to encourage faculty to implement the tools and 
techniques necessary for distance education (Graham & Jones, 2011; LaPrade et al., 2011). 

Conceptual understanding: Framework Organization. Scholars agree that faculty are 
concerned that creating distance courses requires an increased amount of time and resources 
(Cook et al., 2009; Graham & Jones, 2011; Orellana, 2006; Orr et al., 2009; Sener, 2010; 
Singleton & Session, 2011). Creating systems that are simple to implement will encourage 
faculty to create online courses (Graham & Jones, 2011; Singleton & Session, 2011). 

Assessment practices also demand that pedagogical quality systems be simplified to promote 
faculty willingness to use them to create and assess distance courses (Ohia, 2011). Creating an 
instructional design system that is organized and simple to navigate was important to meeting the 
guidance supported by literature. 

Conceptual understanding: Comprehensiveness. Smith (2011) observed that external 
pressure require institutions of higher education develop comprehensive models addressing 
quality, institutional assessment, and quality instruction. Some existing models do exist to 
support quality in distance education but scholars claim that these models do not fully address 
quality from programmatic assessment to instructional delivery (Battin-Little, 2009; Pollacia & 
McCallister, 2009; Westerfelt, 2011). In the development of appropriate model for quality 
instruction, Forsythe, et al. (2010) suggested that a framework be both flexible and provide a 
comprehensive series of choices to support faculty. 

Analysis of Research Question 1 . The first research question in this study sought to 
determine the appeal of the instructional delivery framework for distance education. The 
question was stated as, “What are the perceptions and attitudes of faculty in a small university 
about how a proposed distance education quality course delivery framework is appropriate and 
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appealing to them?” In Formative Research appeal is the enjoyment those using the instrument 
experience while using the instructional design framework (Reigeluth & Frick, 1999). While 
Reigeluth and Frick (1999) discuss appeal as distinctly separate from effectiveness and 
efficiency many study participants commented that in order for the framework to appeal to them 
the items must be both effective and efficient. 

Data were gathered following the data analysis procedure described in an earlier section. 
Specific to this analysis, data were marked with ten initial codes. The codes were revisited and 
consolidated into five categories. These categories were analyzed for critical meaning, following 
Lichtman’s (2013) Three Cs Model, and placed into the five concepts or themes discussed 
earlier. 

What follows is an analysis guided by the research question broken down by each version 
of the framework. Each of the three study groups experienced the appropriate version of the 
framework (i.e.: the first study group used version 1). Each version section described what 
worked or appealed, what did not work or did not appeal, and what revisions were made based 
on the information provided by study participants. 

Version 1 . Participants using the first version of the instructional delivery framework 
provided several responses that discussed the how the framework appealed to faculty at the 
research institution. 

What was Appealing. All study participants in the first participant pool commented about 
the concept of Pedagogical Flexibility they experienced in applying the instructional delivery 
framework to the sample case. Participants acknowledged the ability to make a choice about 
what was included in their courses was critical to their use of the instructional delivery 
framework. This sentiment is the pedagogical flexibility concept. Participants felt that having a 
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comprehensive system of delivery items promoted flexibility and better enabled them to choose 
what was needed. Kirkwood and Price (2008) validate this sentiment with the notion that quality 
delivery of distance courses is only possible when faculty members feel free to elect what 
elements will be in the course. Mancuso (2009) concurred that faculty members are better 
equipped to determine what is needed than a prescribed system of directives. Participants in the 
study remarked that being able to choose empowered them to make wise choices that promoted 
the rigor of the course for student achievement. When asked about the appeal of the framework, 
participant 1001 commented, “the academic freedom of piecing together whatever I want to in 
the course with the tension of making sure my students get the rigor they’re supposed to get.” 

Statements and remarks from the first participant pool included items relating to the 
concept of Teaching and Learning Quality that show the framework as a pedagogy-first approach 
to delivering distance education. Participant 1002 commented specifically to this concept, “I 
think the framework helps pedagogy come first.” The same participant later stated, “it isn't 
about the technology; it is about ‘what are your learning goals?’ and what technology will 
facilitate that.” Graham and Jones (2011) recommended a system be incorporated to abate the 
trepidations of faculty about distance learning. This version framework provides such a system 
whereby pedagogical practice is incorporated with technology. Other participants in this sample 
set also remarked that the technology and distance modality came secondarily to sound teaching 
and learning practices that promote quality included two statements about how the framework 
demonstrated that quality online education is grounded in good teaching habits. 

The next concept addressed by the group was Support and Necessary Skills. The first 
version of the instructional delivery framework (Appendix F) contained three primary categories 
of item support. First was the item name (i.e.: video lecture, discussion board, quiz). Second 
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was an operational definition or explanation of the specific item. Finally, the framework 
contained series of recommendations for use related to the specific item (i.e.: directions or 
suggestions related to the video lecture). This design was purposeful and participants 
commented through the data collection phases about the support and guidance the framework 
provided. Participants in this sample set commented about specific items with remarks that 
indicated some measure of new knowledge, new skills, and new ability to offer that specific tool 
or interaction in their courses indicating the concept of Support and Necessary Skills. These 
comments ranged from use of specific instructional elements such as video lecture, quizzes, 
discussion boards, and tests to non-specific observations about the recommendations for use and 
explanation sections of the framework (Appendix F). Al-Salman (2011) recommended that in 
order to increase faculty practicing quality distance courses that skills development was 
essential. During the interview portion of data collection, participant 1004 remarked, “it 
externalizes and concretizes a lot of chaos and ambiguity that may be lying internally.” The 
comment suggests that framework provided skills guidance for using specific tools or methods 
that might be unfamiliar or confusing to a faculty member teaching an online course. 

The concept of Organization was mentioned in only a few remarks related to the appeal 
of the instrument. The instructional delivery framework was designed using a hierarchical 
format that included references to separate sections concerning course materials, student-student 
interactions, faculty-student interactions, learning activities, and student assessments (Appendix 
F). Participants remarked that the framework was simple and straightforward in its hierarchical 
design. 

Framework Organization, the final concept, was most often discussed in conjunction with 
the comprehensiveness of the instrument. Members of the first study group experienced the 
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framework as a paper version. Participants discussed the complexity of the instructional delivery 
framework and tempered their perception of complexity with the need for the instrument to be 
thorough. The participants also expressed that through a short time of reviewing the instrument 
it became easier to use. The instructional delivery framework’s comprehensiveness was 
expressed in two different perspectives as well. First that the framework was so comprehensive it 
could be easily adapted for hybrid and face-to-face modalities. 

What was not Appealing. Elements that did not appeal to the participants of the first 
study group fell only within one concept: Teaching and Learning Quality. Important to teaching 
and learning quality in distance education is that no single rigid model of instruction works with 
all courses, programs, or instructors (Kirkwood & Price, 2008; S. A. Lei & Gupta, 2010; 
Mancuso, 2009; Schuck et al., 2008; Yair, 2008). This is similar to the concept of Pedagogical 
Llexibility. However, in this concept comments included the need for the framework to account 
for differences in teaching and learning between disciplines and reverence to the institutionally 
specific measures of quality. These items were not present in the first version and several 
comments were made that these were required in order to better support quality instruction. One 
participant commented that recent discipline-specific accreditation aspects of course design were 
addressed and suggested that links to existing departmental practices would be helpful if 
included. Another participant observed that within different disciplines there are standards for 
teaching and learning and including a link to established guidelines would ease the use of the 
framework in conjunction with the discipline specifics. 

Most of the participants in the first study group observed that no synchronous or live 
interaction between students was addressed. Participants suggested that while this was absent in 
the framework the addition of interactivity would better enable students and faculty to remain 
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connected to the course, content, and faculty member. Tanner (2009b) concluded that having 
higher levels of interactivity benefited students and enhanced student outcomes for online 
classes. Including synchronous elements for the course would also diminish the perception or 
reality of isolation in online courses (S. A. Lei & Gupta, 2010). 

Revisions to Increase Appeal. While revisions to the instrument took place based on 
considerations of all the research questions, each change is attributable to a specific research 
question (Appendix G). Table 2 provides a summary of changes made to the first version of the 
instructional delivery framework concerning appeal. With regard to the first research question 
addressing the appeal of the instructional delivery framework, two changes were made. The first 
change was the addition of Departmental and Institutional Specific elements to the Course 
Design page of the framework. Columns were added to the second version of the instructional 
delivery framework (Appendix F) that provided links to or placeholders for established 
departmental and institutional procedures, policies, and statements. Empirical evidence 
supported this revision. Quality and assessment practices for institutions must address the 
pedagogical and quality standards, practices, and policies of the individual institutions 
(Pratasavitskaya & Stensaker, 2010). 

The second change requested by the participants in the first study group that was 
attributed to the appeal aspect of the instrument was the addition of Synchronous Electronic 
Communication in the Faculty-Student Interaction section of the framework. This change was 
validated through research that suggested both faculty and students express a desire for some 
synchronous interaction in an online courses (Graham & Jones, 2011; Singleton & Session, 

201 1). The change included evidence-based recommendations for use. The first 
recommendation was to incorporate the use of synchronous text conversations that aids student 
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in constructing knowledge, negotiation, support, and group processing around course tasks 
(Maushak & Ou, 2007). The second recommendation involved the use of live audio or video 
communication that was found to reduce perceived barriers in largely asynchronous online 
courses (Huang & Hsiao, 2012). The final recommendation advises faculty to establish 
parameters, rules of conduct, and expectations for students with regard to any synchronous 
communication within the online courses (Huang & Hsiao, 2012). 

Table 2 


Summary of Revisions to Version 1 Concern ing Appeal 


Framework Item 

Change 

Research Question 

Course Design 

Add Departmental and Institutional 
specifics to Course Design page. This 
includes both instructions and links to 
college web documents. 

RQ1: Appeal 

Faculty-Student 

Interactions 

Added Synchronous Electronic 
Communication element. Added 
description and recommendations for 
use. 

RQ1: Appeal 


Version 2. Revision of the instructional delivery framework in the specific areas of 
appeal and combined with revisions in the areas of effectiveness and efficiency, analyzed, below 
produced a new version for consideration. Members of the second study group also had 
statements and remarks that responded to the appeal research question. 

What was Appealing. Data gathered from the participants in this study group included 
remarks aligned with the concept of Pedagogical Flexibility. Participants in this study group felt 
that the comprehensive instructional choices and recommendations for use enabled them to 
freely select the most appropriate, effective, or convenient instructional method for the course. 
Commentary included comfort with agreeing or disagreeing with either the tool or the technique 
as appropriate for the course or instructor. Participant 2003 stated, “It was very easy to kind of 
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frown upon [an item] and say nope that is not the option for me.” The same participant also 
remarked that while some options did not fit it was necessary to see the items fully explained. 
Kirkwood and Price (2008) posited that quality instruction requires faculty to freely select 
instructional methods, as they deem appropriate. 

The second participant group provided information that fell into the concept of Teaching 
and Learning Quality. Participants working with the second version presented information that 
relates to the teaching and learning quality concept. Scholars suggest that providing faculty 
members a system that allows for flexibility also promotes quality in teaching and learning 
(Kirkwood & Price, 2008; S. A. Lei & Gupta, 2010; Mancuso, 2009; Schuck et al., 2008; Yair, 
2008). Participants in this sample group commented that the items, descriptions, and 
recommendations for use allowed them to adhere to or adopt empirically-based best practices to 
meet the learning objectives of the course. Scholars concur that faculty members desire 
evidence-based suggestions for instruction and often seek to validate their ideas about teaching 
and learning practices with research (Oermann, 2007; Pratasavitskaya & Stensaker, 2010; Yair, 
2008). 

Participants in this group provided information falling into the concept of Support and 
Necessary Skills. The participant group commented favorably about the recommendations as 
guides, suggested activities, and empirically based best practices. Participant 2002 suggested 
that with that improvement it would provide a basis for discussion more support with the correct 
technical support staff member. All participants felt that additional instruction was needed at the 
item level of the framework to support faculty working independently. Specifically, participant 
2001 felt the language and wording were unclear for non-technical faculty. Additionally, 
participants 2003 and 2004 described a need to see how different instructional elements operated 
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within the context of an online course to better inform their decision making about how it was 
used or whether to use it at all. 

Participants working with version two of the instructional delivery framework provided 
responses that were coded and categorized in the concept of Framework Organization. 
Participants in this study group also commented on the organization of the instructional delivery 
framework. The participant group noted the header color-coding was useful as they worked 
through the sample case to stay on track. One participant commented that the organization was 
helpful in keeping the development of the course on track given that it can often be chaotic. The 
sequence of the items was also helpful to some participants. Specifically, participant 2004 
stated, “I like the way the framework foregrounds the description, mission, purpose, and core 
curriculum. Then takes the more pragmatic application pieces and puts them later.” 

The second study group remarked about the concept the concept of Comprehensiveness 
as well. All participants compared the comprehensiveness of the framework to the complexity of 
the instrument. Participants suggested that while exposure to the instrument was sufficient a 
detailed set of instructions for the framework would make it a more appealing and effective tool 
for developing and delivering online courses. This group was offered the framework in both web 
format and paper format and remarked that while not selected the web format might have been 
easier to navigate. Despite the complexity of the instrument, the participants noted that the task 
of delivering distance education courses is a complex one and requires a thorough and 
comprehensive model for faculty to follow. 

What was not Appealing. Participants in the second study group provided information 
indicating an unappealing aspect in only one concept; Comprehensiveness. All participants 
compared the comprehensiveness of the framework to the complexity of the instrument. 
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Participants suggested that while exposure to the instrument was sufficient a detailed set of 
instructions for the framework would make it a more appealing and effective tool for developing 
and delivering online courses. This group was offered the framework in both web format and 
paper format and remarked that while not selected the web format might have been easier to 
navigate. Despite the complexity of the instrument, the participants noted that the task of 
delivering distance education courses is a complex one and requires a thorough and 
comprehensive model for faculty to follow. Research indicates that when faculty members are 
unfamiliar with the technological tools surrounding online learning these items become a barrier 
to adoption of quality techniques in the distance classroom (Singleton & Session, 2011; Tabata 
& Johnsrud, 2008). 

Revisions to Increase Appeal. Revisions for the second version of the instructional 
delivery framework included two changes dealing the appeal of the instrument (Appendix G). 
Table 3 provides a summary of changes made to the second version of the instructional delivery 
framework concerning appeal. First, and in conjunction with changes made relating to the other 
research questions, the participants felt a web-version of the framework would be more 
appealing and a web-only format was created. Research suggests that the development of web- 
based tools will better equip institutions of higher education to monitor and deliver quality online 
courses than traditional tools and processes might do when applied to online learning (Postek et 
al., 2010; P. S. Smith, 2011). Second, instructions were added to the framework to help 
acclimate and prepare faculty members to use the framework without leaving the 
comprehensiveness of the instrument to overwhelm the users. Increasing faculty familiarity with 
the tools, techniques, and processes surrounding distance education removes barriers to adoption 
and implementation of quality courses (Singleton & Session, 2011; Tabata & Johnsrud, 2008). 
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Table 3 


Summary of Revisions to Version 2 Concerning Appeal 


Framework Item 

Change 

Research Question 

Format 

Converted the framework to a web- 
only format with hyperlinking 
navigation between sections. 

RQ1: Appeal 

General Framework 

Added Framework Instructions to a 
new first page. 

RQ1: Appeal 


Version 3. Participants in the third study group provided data regarding the appeal of the 
instructional delivery framework in only four of the five concepts. Namely, the data contributed 
to the concepts of Pedagogical Flexibility, Support and Necessary Skill, Framework 
Organization, and Comprehensiveness. 

What was Appealing. Participants in the third study group expressed that the 
instructional delivery framework contained pedagogical flexibility. Participant 3002 remarked 
that due to the instructions and very little additional guidance that there was considerable 
freedom to make the elections deemed appropriate and useful. Additionally, participant 3001 
commented on the inclusion of an ongoing framework additions, changes, and comments 
protocol, discussed in the second research question analysis, enabled the faculty to freely suggest 
practices they felt needed to be included. As suggested earlier, scholars recommend that faculty 
be allowed to select pedagogical elements freely in order to accurately and effectively meet 
quality and instructional objectives (Kirkwood & Price, 2008; McLawhon & Cutright, 2012; 
Mustafa & Dalen, 2006; Selwyn, 2011; Yair, 2008). 

Comments addressing the concept of Support and Necessary Skills indicated that 
participants characterized the revised framework with additional instructions as easy to use, links 
to additional resources, evidence-based best practices, and very clear examples that increased 
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familiarity with the modality and technique of online instruction. Participant 3001 stated, “You 
gave provided links to other sources. You’ve referenced things. So, it is very user-friendly to 
use.” Faculty members who are more familiar with the tools and techniques of delivering online 
instruction perceive less barriers to producing quality distance courses (Singleton & Session, 
2011; Tabata & Johnsrud, 2008). 

Participants provided data that contributed to the Framework Organization concept as 
well. Both participants expressed, that even working independently, the framework was laid out 
clearly and the systematic design made is simpler to follow and apply to the sample case. The 
participants commented that the organization and categorization of the tools aided in considering 
possibilities for the course being designed. 

Finally, participants remarked about the concept of Comprehensiveness in the interviews 
and focus group. Discussions about the third version of the framework stated that the 
instructions reduced the complexity and allowed the participants to focus on the comprehensive 
set of considerations. Participants remarked that the instrument was detailed and thorough. The 
thoroughness provided participants with much to consider regarding what could be included in 
an online course. In one part of the discussion, participants stated that the collection of items and 
techniques was far a far greater collection that might be developed on one’s own making the 
resource very appealing as a design and ongoing delivery instrument. 

What was not Appealing. Participants in the third study group did not indicate any item, 
element, or portion of the instructional delivery framework as unappealing. Participant 3002 
commented that the ongoing changes protocol was cumbersome with the requests for evidence- 
based suggestions and indicated a desire to submit an idea for consideration. Contrary 
commentary from another participant suggested that the in-depth protocol helped maintain 
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quality, increase faculty awareness of teaching and learning scholarship, and prevent the 
framework procedure from being bogged down researching suggestions that should come 
researched. The streamlining of the change suggestion protocol was left unchanged in the 
framework. 

Revisions to Increase Appeal. The third group of participants made no recommendations 
for improvements to appeal. Areas of adequacy, inadequacy, and suggestions for future research 
with this framework follow in subsequent sections of this document. 

Summary of Research Question 1. The first research question is stated, “What are the 
perceptions and attitudes of faculty in a small university about how a proposed distance 
education quality course delivery framework is appropriate and appealing to them?” Through 
data collection in three distinct study groups, it was determined that the appeal of the framework 
was that the instrument allow for flexibility and freedom to determine what teaching and learning 
techniques, tools, and inclusions amount to quality in an distance education course. Scholars 
concur that when flexible models of instruction exist that enable faculty choice about pedagogy 
result in higher quality course delivery (Kirkwood & Price, 2008; S. A. Lei & Gupta, 2010; 
Mancuso, 2009; Schuck et ah, 2008; Yair, 2008). Appeal of the framework included that the 
instrument placed pedagogy before technological tool which aids faculty in remaining focused 
on the learning objectives, course outcomes, and their own experiences instructing students. 
Graham and Jones (2011) concluded that putting instruction first encourages faculty participation 
in online learning. Lastly, that a comprehensive, thorough offering of choices, guidance, and 
instructions increases the appeal of the instrument and thus encourages use. In the following 
subsections, the areas of effectiveness and efficiency are addressed to provide a complete picture 
of the adoption of the instructional delivery framework. 
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Analysis of Research Question 2. The second research question in this study sought to 
determine the effectiveness of the instructional delivery framework for distance education. The 
question was stated as, “What are the perceptions and attitudes of faculty in a small university 
about how a proposed distance education quality course delivery framework will meet their 
needs for effectiveness in delivering quality online courses?” In Formative Research 
effectiveness is determined by how well the instructional design model or framework meets the 
goals in a given setting, situation, or course (Reigeluth & Frick, 1999). 

Data were gathered following the data analysis procedure described in an earlier section. 
Specific to this analysis, data were marked with ten initial codes. The codes were revisited and 
consolidated into five categories. These categories were analyzed for critical meaning, following 
Lichtman’s (2013) Three Cs Model, and placed into the five concepts or themes discussed 
earlier. 

What follows is an analysis guided by the research question broken down by each version 
of the framework. Each of the three study groups experienced the appropriate version of the 
framework (i.e.: the first study group used version 1). Each version section described what 
worked or was effective, what did not work or was not effective, and what revisions were made 
based on the information provided by study participants. 

Version 1 . Participants using the first version of the instructional delivery framework 
provided several responses that discussed the how effective the framework was for faculty in 
meeting instructional and quality goals at the research institution. 

What was Effective. Participants in the first study group provided information about the 
effectiveness of the framework that supported the concept of Pedagogical Flexibility. This 
included commentary from participants that the instructional delivery framework, when applied 
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to various courses, enabled them to make different choices than were selected for the sample 
case as part of the study. The decision to alter pedagogical methods based on different course 
objectives is critical to promoting quality instruction and programming at colleges and 
universities (Ching- San et al., 2009; Mustafa & Dalen, 2006; Yair, 2008). Participants 
discussed that the framework allowed them to use more, less, or different interactions in various 
courses at several times during the data collection phases. Participants felt free to agree and 
disagree with elements of the instructional framework either because of their experience, lack of 
familiarity with the item, or discomfort with the item in some way. Yair (2008) suggested that 
this flexibility to agree or disagree with teaching techniques is part of the academic freedom 
faculty must enjoy in order to ensure a quality course, program, and institution. One specific 
comment illustrates much of the individual and group discussion on this point. Participant 1004 
stated, “Here is what we think what might be important to developing a good online course. 
Then I can say I agree or I disagree versus searching around in my own experiences about what 
would be useful or what is good and bad.” 

Support for faculty teaching at a distance figured in the participant remarks. Lee, at al., 
(2010) and Graham and Jones (2011) all posited that support systems that encourage faculty, 
provide guidance, and contribute to online teaching skills are essential to providing quality 
distance education. Participant 1002 commented that, “when I first started there wasn’t a 
framework so as newer technology was coming in, you were determining how you could apply 
it.” This comment was followed by, “In my view, this framework already maps that out.” The 
participant was referring to the instructional delivery framework’s support mechanics for using 
technology in a distance course. Within the study groups participants expressed varying degrees 
of familiarity with technology and distance education. Faculty who are more familiar with the 
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uses of education technology and online learning are more likely to attempt or expand their uses 
of these forms of teaching (Singleton & Session, 2011; Tabata & Johnsrud, 2008). Participant 
1004 commented most significantly on this area and revealed a very limited experience with 
online courses and from experience and dialogue developed a resistance to teaching at a distance. 
Upon completion of the application of the instructional delivery framework to the sample case, 
the same participant said, “. . .it doesn’t seem like the online classes that I am familiar with or that 
people I know follow this [framework]. If they did I wouldn’t have such a negative perception 
of online learning.” This same participant continued, “If I were to do an online course, I’d like 
to follow this sort of thing. It has everything I can imagine that I would need to be in it.” The 
framework then provided a support system that not only developed skills and provided support 
but also encouraged faculty who were unfamiliar with distance education to attempt a course or 
have a more positive outlook on the modality. 

Participants in the first study group provided information supporting the concept of 
Teaching and Learning Quality. Study group members commented on the notion of multiple 
forms of assessing student learning. Participant 1003 stated, “I think the assessments were 
helpful in coming up with different ways to assess the students.” The frameworks inclusion of 
varying types of assessment allows faculty to better match learning goals with measurement of 
student outcomes. Quality in teaching and learning for institutions of higher education needs to 
create strong links between learning objectives, teaching practices, and assessment of students 
(Bers, 2008; Ziliukas & Katiliute, 2008). 

The concept of Framework Organization did appear in the coded dialogues of the 
participants. Commentary included that the instrument was thoughtful in design and followed a 
logical workflow. Participant 1003 stated, “I think it is nice that it is laid out the way it is 
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because it is laid out the way you might expect your syllabus to be laid out.” All participants 
commented that the framework described and ordered the activity by key functional area and that 
following along was convenient and straightforward. 

Participants in the first study group offered little data regarding the concept of 
Comprehensiveness in relationship to the effectiveness of the instructional delivery framework. 
Participants felt that comprehensiveness of the model was expressed through the evidence-based 
item details suggesting that it was academically sound that supported making effective 
pedagogical decisions. Scholars and research indicated that faculty members are more effective 
when empirical evidence for instructional delivery techniques, methods, and tools are used 
(Oermann, 2007; Pratasavitskaya & Stensaker, 2010; Yair, 2008). 

What was not Effective. Participants in the first study group did not indicate any aspect 
of the instructional delivery framework was not effective. While revisions were recommended 
as they relate to appeal and efficiency of the instrument, this study group provided no data 
indicating ineffective elements to be changed. 

Revisions to Increase Effectiveness. No effectiveness revisions were included in the 
revision from version one of the instructional delivery framework to version two. 

Version 2. Participants using the second version of the instructional delivery framework 
provided several responses that discussed the how effective the framework was for faculty in 
meeting instructional and quality goals at the research institution. Participant contribution and 
coded data supported three of the five mentioned concepts: Pedagogical Flexibility, Support and 
Necessary Skills, and Teaching and Learning Quality. 

What was Effective. Participants provided information that supported the Pedagogical 
Flexibility concept. These Participants remarked about pedagogical flexibility and the ability to 
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choose instructional methods. Two participants of the four commented that the instructional 
delivery framework allows for expansion and should include the ability for faculty to add to it 
based on their experiences, new ideas, new tools, or experimentation with the instruction. 
Mancuso (2009) suggested faculty members are better equipped to make recommendations about 
instruction because of their expertise in the discipline and ability to convey that expertise. 
Participants agreed that the capacity for the framework to be fluid and scalable with faculty input 
was essential to using the tool in the delivery of quality distance education courses. The 
suggestions of the participants were to add the ability of making suggestion to the instrument. 
Distance education quality is influenced by the ability of faculty to freely select options 
appropriate to them and discard those that do not fit their needs (Kirkwood & Price, 2008; Yair, 
2008). Participants in this study group remarked that without the ability to choose would cause 
dissatisfaction amongst faculty. Participant 2002 in response to a discussion about how distance 
education can become rigid remarked, “it can infringe on academic freedom if the college says 
online courses must be done in this way.” Another participant agreed at a later point suggesting 
that a rigid structure or prescribed method of instruction would inhibit the willingness of faculty 
to attempt new techniques as they or their courses evolve. Participant dialogue regarding the 
instrument demonstrated that the ability to pick and choose items, methods, techniques, or tools 
based on the objectives was important and that the instructional delivery framework did not 
hinder that freedom and flexibility. 

With regard to the Support and Necessary Skills concept, participants offered information 
that indicated the effectiveness of the instructional delivery framework. Participants in this study 
group described the second version of the framework as helpful for faculty new to instructing 
online, the profession, or the institution. This included one remark about the framework being a 
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self-diagnosing tool during course delivery if an interaction was not meeting expectations. 
Participants remarked that the framework could potentially serve as a resource for changing the 
interaction, using an alternative recommendation for use, or determining if the implementation 
was flawed in some manner. 

The Teaching and Learning Quality concept also explained data that was collected 
regarding effectiveness. The participants in the group suggested three additions to the 
framework in the Student Assessment area of the instructional delivery framework including 
drafted work, concept documents for assignments, and collaborative writing (Appendix F). The 
existing items of formatted and unformatted written assignment categories addressed these 
recommendations and the participants agreed they could easily adopt the existing assessment 
devices. The adaption of items on the framework shows the malleability of the instrument for 
faculty in adherence to literature about the using non-rigid models (Graham & Jones, 2011). 

What was not Effective. Ineffective elements of the framework fell into only the Support 
and Necessary Skills concept derived from the data collected. Participants envisioned the 
framework, as constructed or with the additional item-level instructions, as training tools that 
served in-person or self-study programs. Participant 2002 felt that additional step-by-step 
instructions could bridge the gap between seeing the item in the framework and implementing in 
a course. This included both technical instructions as well as implemented examples from other 
faculty members. Participants commented heavily about the inclusion of faculty-use examples 
into the instrument and also suggested that a submit changes or additions protocol be included. 
The participants, to build in ongoing effectiveness and relevance of the framework and serve as a 
source for examples, intended the change submission protocol. Assisting faculty through 
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development and instructional opportunities promotes quality online instruction (Al-Salman, 
2011; Lee et al., 2010). 

Revisions to Increase Effectiveness. Three revisions to the second version were sought to 
improve the effectiveness of the instrument for faculty use in distance education (Appendix G). 
Table 4 provides a summary of changes made to the second version of the instructional delivery 
framework concerning effectiveness. Each of these changes related to the Support and 
Necessary Skills concept. Two of the three revisions to this version were done at the interaction 
item level in the course materials, faculty-student interactions, student-student interactions, 
learning activities, and student assessment sections. The third revision was added at the 
instrument level and was the addition of a change submission protocol. 

The first revision was the addition of Faculty Use Examples throughout the subsections 
of the instructional design framework. Research suggests that distance education quality is more 
likely to occur when faculty members’ online teaching skills are supported and expanded 
through a system of instruction and encouragement (Al-Salman, 2011; Lee et al., 2010). Within 
each section a column was added and where faculty use examples existed they were linked in the 
electronic version of the document. In many cases, existing examples did not exist and could not 
be included. For the purpose of the research a placeholder was put in the instrument to indicate 
that it is awaiting proper examples. 

The second revision was the addition of Setup Instructions at the interaction item level. 
Similar to faculty use examples, the intention was to provide faculty members live, just-in-time, 
support for either understanding or implementing the tool or technique in their online course. 
Rather than be a use example, the setup instructions build additional online teaching skills 
helping faculty deliver a higher quality distance course (Al-Salman, 2011; Lee et al., 2010). This 
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revision included the addition of another column in the framework and links to existing 
instructions provided by the learning management system company or service company 
responsible for a tool that could be used with that particular item. Linked instructions include 
both video and text-based instructions. 

The final revision based to enhance effectiveness of the framework is the inclusion of a 
change protocol by which faculty can submit ideas, comments, suggestions, additions, or 
changes to the existing framework. Faculty members are closest to instruction and therefore 
better equipped to determine how and what is needed to meet the learning and quality objectives 
in distance education (Kirkwood & Price, 2008; Mancuso, 2009). These changes involved the 
creation of a protocol for users of the framework to follow with a hyperlink connected to email. 
Emails would be sent to the administrator responsible for maintenance of the instructional 
delivery framework for consideration and inclusion. The change submission protocol included 
instructions to provide detailed description of the change or suggestion along with empirical 
evidence supporting the submission. 

Table 4 


Summary of Revisions to Version 2 Concern ing Effectiveness 


Framework Item 

Change 

Research Question 

Interaction Sections 

Added Faculty Use Examples or 
placeholders where no examples exist 
yet. 

RQ2: Effectiveness 

Interaction Sections 

Added Setup Instructions via linking to 
video or documentation. 

RQ2: Effectiveness 

General Framework 

Added Submit Changes protocol to a 
new last page of the framework with 
hyperlink to email a coordinator. 

RQ2: Effectiveness 


Version 3. Participants using the third version of the instructional delivery framework 


provided several responses that discussed the how effective the framework was for faculty in 
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meeting instructional and quality goals at the research institution. Coded data fell into three of 
the five concepts: Pedagogical Flexibility, Support and Necessary Skills, Teaching and Learning 
Quality, and Framework Organization. 

What was Effective. Members of study group three provided information that supported 
the concept of Pedagogical Flexibility. The participants remarked that the instrument provided a 
list of resources, tools, and techniques for consideration that aided them in meeting the 
instructional goals. Participant 3002 indicated that the list of resources was not dissimilar from 
walking into a physical classroom and taking inventory of what was available to meet the course 
objectives. In this last version of the instrument a set of protocols allowed for faculty input with 
regard to new techniques, suggestions, or changes to foster more freedom and flexibility. The 
participant continued to positively comment that the protocol required some evidence-based 
thinking and not only a recommendation for changes. One participant in this study group 
expressed that the instructional delivery framework was a living document where the faculty 
member could revisit their original thinking and revise it based on what was seen in the either the 
framework or the course. 

Participants also provided information about the instructional design framework that fits 
the concept of Support and Necessary Skills. Participant 3001 concurred offering that the items 
and recommendations for use provided a base from which instruction could be adapted to meet 
the needs of the faculty and courses. This version, based on effectiveness revisions from version 
two, also included visual examples of items in use within a course or support document. Both 
participants remarked that seeing these examples within the framework better enabled them to 
make effective choices for their courses. Participant 3002 commented on the increased detail, 
“there was support material that was helpful to imagine how I might use that and how I might 
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feel better informed so I can make a better decision or not.” Finally, both participants in this 
study group indicated that not only was the framework helpful for the initial delivery of the 
course but as a reference tool throughout that could assist them in overcoming problems in the 
course with interactions or tools. Mancuso (2009) proffered that when adequately supported, 
faculty are best equipped to make determinations about delivery methods, course inclusions, and 
pedagogical practice. 

The Teaching and Learning Quality concept was represented in the data collected from 
the third study group. Participants in the third study group remarked that the third version of the 
instructional delivery framework placed pedagogy first, ahead of technology. Participant 3002 
stated, “the framework works for me as a delivery guide to know what tools I have. In that sense 
not different from trying to figure out what things exist in a physical classroom.” The participant 
remarked that this enabled putting the course goals and vision of instruction first above tools. 
Kupczynski, et. al. (2012) concurred that quality in distance education is achieved when 
pedagogy takes the primary role in constructing an online course. 

Lastly, participants provided comments that address the concept of Framework 
Organization. Participants in this subset remarked that the organization of the instructional 
delivery framework followed a natural flow of working with an instructional designer to develop 
an online course. Research suggest that in order for distance education to effectively meet 
quality objectives partnerships between faculty experts and instructional design experts should 
be fostered (Pratasavitskaya & Stensaker, 2010). This data point shows that the instrument can 
make working independently more effective by following a natural workflow of partnering with 


a designer. 
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What was not Effective. The third group of participants provided no recommendations 
about improving the effectiveness of the framework. No Data were collected that indicate 
ineffective aspects of the instrument. 

Revisions to Increase Effectiveness. The third group of participants made no 
recommendations for improvements to appeal. Areas of adequacy, inadequacy, and suggestions 
for future research with this framework follow in subsequent sections of this document. 

Summary of Research Question 2. Effectiveness is a factor in the development of 
instructional design model, theory, or framework that is determined by how well the instrument 
assists stakeholders in accomplishing the curricular goals (Reigeluth & Frick, 1999). The second 
research question determined the attitudes and perceptions of faculty regarding the effectiveness 
of using the instructional delivery framework for implementing quality online courses. 
Participants in each of the research groups provided information that improved the instrument to 
a final version that met the participant needs for effectiveness. 

The instrument was effective in meeting the needs for pedagogical flexibility and 
freedom to choose what interactions, items, techniques, and tools would best meet the course, 
student, and faculty needs of the course. When faculty members are left to determine the best 
methods of meeting course objectives, distance courses are better able to meet quality guidelines 
(Ching- San et ah, 2009; Mustafa & Dalen, 2006; Yair, 2008). Participants remarked that the 
instruments built in support helped them choose and implement their choices in an online course 
and recommended that more detailed examples and instructions be incorporated. 

It was also determined that the detailed instructions would support faculty regardless of 
experience level with distance education. Supporting faculty in the development of skills and 
familiarity with distance learning tools is central to offering quality online courses and programs 
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(Singleton & Session, 2011; Tabata & Johnsrud, 2008). The instrument was shown to be 
organized in a way that fit natural and established workflows for faculty and their instructional 
design partners providing an effective framework for either working together or collaboration. 
Participant 1004 specifically commented that the organization and inclusions of this framework 
changed his opinion of distance education provided faculty used this instrument. In the 
following subsection, the area of efficiency is addressed to provide a complete picture of the 
adoption of the instructional delivery framework. 

Analysis of Research Question 3. The third research question in this study sought to 
determine the effectiveness of the instructional delivery framework for distance education. The 
question was stated as, “What are the perceptions and attitudes of faculty in a small university 
about how a proposed distance education quality course delivery framework will meet their 
needs for ease of use in delivering quality online courses?” In Formative Research efficiency is 
determined by how easily the instructional design model or framework meets the goals in a given 
setting, situation, or course (Reigeluth & Frick, 1999). In this study efficiency questions relate to 
how easily, simply, or conveniently the instructional delivery framework was for faculty 
members. Several respondents commented that discussions about efficiency intersected with 
questions about appeal and effectiveness stating that the instrument would not appeal nor be 
effective if efficiency did not exist. 

Data were gathered following the data analysis procedure described in an earlier section. 
Specific to this analysis, data were marked with ten initial codes. The codes were revisited and 
consolidated into five categories. These categories were analyzed for critical meaning, following 
Lichtman’s (2013) Three Cs Model, and placed into the five concepts or themes discussed 


earlier. 
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What follows is an analysis guided by the research question broken down by each version 
of the framework. Each of the three study groups experienced the appropriate version of the 
framework (i.e.: the first study group used version 1). Each version section described what 
worked or was efficient, what did not work or was not efficient, and what revisions were made 
based on the information provided by study participants. 

Version 1 . The study group using the first version of the instructional delivery 
framework provided responses to interview and focus group questions that were categorized into 
the concepts of Pedagogical Flexibility, Teaching and Learning Quality, Support and Necessary 
Skills, Framework Organization, and Comprehensiveness. 

What was Efficient. Participants in the first study group provided responses that met the 
conceptual need for Pedagogical Flexibility. Participant 1001 provided numerous statements that 
the efficiency of the system was found in the choices presented for instruction. Participants 
remarked that the choices enabled them to quickly and simply extract what was needed to meet 
the course objectives and implement this in the course. 

Comments about Pedagogical Flexibility were closely related to statements from 
participants about Framework Organization. Participants indicated that the organization of the 
framework made it clear what items, interactions, and aspects of a distance course should be 
considered in one location. Participants also remarked that the hierarchical organization of the 
instrument made it simple and convenient to locate items. 

Likewise, participant data about the Teaching and Learning Quality concept were closely 
related to Pedagogical Flexibility and Framework Organization. Participants commented that the 
instrument gathered many types of interactions into one place making it easy to consider both 
what and how to implement items to meet teaching goals. Participant 1003 specifically 
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commented that the description and recommendations for use provided queues to remind faculty 
about what to do or what was needed without having to do extensive searching or reflection thus 
improving the teaching quality more efficiently. 

Data gathered from participants about the Comprehensiveness concept closely 
interrelated with the other four concepts. All participants commented that the data thoroughness 
of the instructional delivery framework enabled them to quickly see a comprehensive gathering 
of teaching tools and techniques. The repository aided faculty to easily locate and implement 
appropriate items for distance education that would allow them to construct a course that met 
quality expectations. 

What was not Efficient. Participants in the first study group made comments about what 
was not efficient in terms of the Framework Organization and Support and Necessary Skills. 
Participants commented and agreed that the comprehensiveness of the framework was adequate 
and suggested that efficiencies could be found by combining similar items. Participants felt that 
while not a detractor from using the framework it added to a sense of overwhelming that could 
be prevented. Specifically, the participants felt that in the course materials section items of 
similar or same nature could be consolidated and the descriptions altered to assist faculty when 
using that item. Participants agreed that the complexity of the document could also eased with 
the use of color-coding. Color codes, as suggested by Participant 1004, could be assigned to 
items in the Course Design section of the framework and the same color used on the related table 
found later in the framework (Appendix F). Finally, participants agreed that the paper format of 
the first version was adequate but that a web-based format might provide some efficiency. 

Revisions to Increase Efficiency. Based on the data about inefficiencies in the 
instructional delivery framework four revisions were made. Table 5 summarizes the changes 
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made to the first version of the framework concerning efficiency or ease of use. First, the Course 
Technology and Course Instructions items in the Course Design section were combined because 
technology instructions in an online course are similar or the same as instructions for success in 
the course. Scholars suggest that a main pedagogical concern with distance courses is that 
students do not have the proper understanding of what is expected or the direction to complete 
basics tasks in the course (S. A. Lei & Gupta, 2010). Providing instructions in a clear and 
concise manner support student success in online courses (S. A. Lei & Gupta, 2010). 

The second revision took items in the Course Materials section and combined them 
where items were the same or extremely similar (i.e.: Video Lecture and Guest Video Lecture). 
Descriptions were modified to account for the slight differences in focus but recommendations 
for use were identical. Scholars agree that quality systems should seek to be less onerous and 
cumbersome for faculty in order to promote ease-of-use and consistent application of quality 
tools (Jordens & Zepke, 2009; Shulman, 2007). 

The third revision was the implementation of color-coding that linked the Course Design 
section components to the corresponding section. For example, the Faculty-Student Interactions 
item on the Course Design page was color-coded green and the corresponding table was coded 
with the same color. Participants indicated that this would make using a paper or web version 
easier. Quality systems for distance education need to find means reduce complexity and 
increase ease-of use in order to encourage faculty adoption of the instruments (Jordens & Zepke, 
2009; Shulman, 2007). 

The final change made to the first version of the framework was the inclusion of a web- 
based format option. The framework was moved from Microsoft Word to Microsoft Excel to 
maintain the table format. Microsoft Excel allows for workbooks to be saved as web pages. 
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Again, making a system of quality more convenient for faculty promotes use and adoption of 
tools that beget better quality distance courses (Jordens & Zepke, 2009; Shulman, 2007). 
Table 5 


Summary of Revisions to Version 1 Concerning Efficiency 


Framework Item 

Change 

Research Question 

Course Technology 

Included Course Technology 
instructions into the Course 
Instructions section. 

RQ3: Efficiency 

Course Materials 

Consolidated items that had similar 
items with the same recommendations 
into one item with a broadened 
description. 

RQ3: Efficiency 

Section Headers 

Added color-coding to Course Design 
page and corresponding with section. 

RQ3: Efficiency 

Format 

Converted the framework from MS 
Word to MS Excel and added Web- 
based format as an option. 

RQ3: Efficiency 


Version 2. The second study group provided responses and data that covered the 
Teaching and Learning, Framework Organization, and Comprehensiveness concepts. 

What was Efficient. Participant discussion provided data that was classified as aspects of 
the Teaching and Learning concept. Members of the study group remarked that the framework 
especially with the inclusion of effectiveness changes including faculty use examples mentioned 
above, is a central process of quality faculty work in distance education. The instrument, as 
participants described, is a mechanism to foster easy sharing of best practices, examples, and 
expansion of distance learning quality approaches. 

Data from this group were also classified under the Comprehensiveness concept. 
Participants in this study group felt that the instrument was comprehensive and provided an easy 


access point for several ideas, tools, and techniques for teaching distance courses. Participants 
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remarked that the instrument was complicated but not confusing to difficult to use. Specifically, 
participant 2002 felt that while complex it was fitting the task of implementing a quality online 
course and that it was appropriately complex given the task. 

What was not Efficient. Participant 2003 described the organization of the instructional 
delivery framework as somewhat out of order and provided a rationale that suggested this was a 
situation specific to the individual. Reigeluth and Frick (1999) recommend that individual 
anomalies should be noted and categorized as situations but not necessarily included in the 
revisions of the theory. 

Revisions to Increase Efficiency. Revisions to the second version of the framework did 
not include changes attributable to the research question of efficiency. 

Version 3. Participants in the third study group provided data that fit within the 
Pedagogical Flexibility, Support and Necessary Skills, Teaching and Learning Quality, and 
Framework Organization concepts. 

What was Efficient. The Pedagogical Flexibility concept was represented in data 
gathered from participants in the context of providing choices and ongoing growth. Participants 
in this study group described the choices contained in the instructional delivery framework as a 
convenient and central repository of items for consideration. The dialogue about this notion 
continued and indicated that within the choices it was easy for faculty members to carefully 
select well described and supported instructional interactions for their respective courses. This 
version of the framework also included a change suggestions protocol. Participant 3001 
remarked that this protocol allowed for the instrument to remain relevant and current easily. 
Whereby faculty could suggest changes following the protocol and those changes, if accepted, 
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could be added to the document without having complex and lengthy data gathering expeditions 
throughout the college. 

Participants also remarked about the Support and Necessary Skills concept. The 
expanded instructions and examples for interaction items provided the basis for participant data. 
Participants in this study group commented that having the recommendations, instructions, and 
examples contained or linked within the instructional delivery framework made using it simple. 
Remarks included that the presence of these within the instrument eliminated the need to locate 
and source multiple references for help. 

Regarding the Teaching and Learning Quality concept, participants remarked that the 
framework provides a singular location for faculty to revisit as the course runs to ensure that 
interactions are going as intended. Participant 3002 suggested that the framework provides a 
type of checklist for delivering distance courses to help the faculty stay on track with how to 
implement the tools in a way consistent with quality practices. 

Lastly, the Framework Organization concept was represented in this participant group 
data. The participants of this group remarked that the hierarchical structure, color-coding, use of 
links, and instructions provided an easy to use system. Participants discussed that the ease of use 
helped them navigate and implement the framework efficiently. 

What was not Efficient. Participants in this study group did not indicate any items were 
inefficient or required change. 

Revisions to Increase Efficiency. The third group of participants made no 
recommendations for improvements to appeal. Areas of adequacy, inadequacy, and suggestions 
for future research with this framework follow in subsequent sections of this document. 
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Summary of Research Question 3. Scholars conclude that the creation of quality 
systems for distance education need to be easily and simple for faculty to use or institutions risk 
a low adoption and use rate (Jordens & Zepke, 2009; Shulman, 2007). This section analyzed the 
third research question stated as, “What are the perceptions and attitudes of faculty in a small 
university about how a proposed framework will meet their needs for ease of use?” The data in 
this area of the study indicated that the efficiency the instructional delivery framework was 
attained through a series of revisions. The framework was moved from a paper-based system to 
a completely web-based format where redundancies were eliminated and organizational aids like 
color-coding was added. Participants indicated that the efficiency of using this instrument was 
that it provided a comprehensive list of interactions with recommendations for use and 
instructions to help faculty implement a quality online course. 

Summary of version changes. Formative Research is an iterative study method where 
versions of instructional design theories, models, or frameworks are put into use by study 
participants (Reigeluth & Frick, 1999). The instrument is continually revised until study 
participants offer no additional substantive changes to improve the appeal, effectiveness, or 
efficiency (Reigeluth & Frick, 1999). In this study, three versions of the framework were used 
until participants provided no data, suggestions, or recommendations that were substantive to the 
appeal, effectiveness, or efficiency of the instrument. Each version of the instructional delivery 
framework are available in Appendix F. 

The data analysis above outlined the attitudes and perceptions of faculty with regard to 
the three research questions. The analysis provided a discussion or what worked, what did not 
work, and revisions that were made. The revisions elements of the analysis are broken down by 
research question and do not give a homogenized version-to-version change description. 
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Tables 6 and 7, below, provide an overview of the changes made to the first and second 
versions, respectively, of the instructional delivery framework based on participant input and 
feedback. Change suggestions were compared to empirical evidence upon analysis and 
implemented in the subsequent version. A full change matrix is available in Appendix G. 

Table 6 


Overview of revisions to Instructional Delivery Framework: Version 1 


Framework Item 

Change 

Research Question 

Course Design 

Add Departmental and Institutional 
specifics to Course Design page. This 
includes both instructions and links to 
college web documents. 

RQ1: Appeal 

Faculty-Student 

Interactions 

Added Synchronous Electronic 
Communication element. Added 
description and recommendations for 
use. 

RQ1: Appeal 

Course Technology 

Included Course Technology 
instructions into the Course 
Instructions section. 

RQ3: Efficiency 

Course Materials 

Consolidated items that had similar 
items with the same recommendations 
into one item with a broadened 
description. 

RQ3: Efficiency 

Section Headers 

Added color-coding to Course Design 
page and corresponding with section. 

RQ3: Efficiency 

Format 

Converted the framework from MS 
Word to MS Excel and added Web- 
based format as an option. 

RQ3: Efficiency 
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Table 7 


Overview of revisions to Instructional Delivery Framework: Version 2 


Framework Item 

Change 

Research Question 

Format 

Converted the framework to a web- 
only format with hyperlinking 
navigation between sections. 

RQ1: Appeal 

General Framework 

Added Framework Instructions to a 
new first page. 

RQ1: Appeal 

Interaction Sections 

Added Faculty Use Examples or 
placeholders where no examples exist 
yet. 

RQ2: Effectiveness 

Interaction Sections 

Added Setup Instructions via linking to 
video or documentation. 

RQ2: Effectiveness 

General Framework 

Added Submit Changes protocol to a 
new last page of the framework with 
hyperlink to email a coordinator. 

RQ2: Effectiveness 


Summary 

The purpose of this Formative Research study was to develop an instructional delivery 
framework for distance education courses at a small private liberal arts university in western 
New York state referred to as Magdalene University in this study. The framework meets 
requirements from accreditation and regulatory agencies, faculty and administration concerns 
about how to design and implement quality distance courses, and maintenance of pedagogical 
flexibility for faculty. Instructional delivery items were well-researched, evidence-based, and 
institutionally sound to provide adequate guidance to experienced and inexperienced distance 
education faculty. 

Chapter 4 presented the analysis and results of the research creating the instructional 
delivery framework. Participants were described to provide demographic information and 
understanding of the data collected. Research questions were restated and used to organize the 
presentation of data analysis and results. Data analysis provided detailed descriptions of what 
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worked, did not work, and required revision until the participants sought no substantive changes. 
The narrative presentation followed the research questions and discussed five concepts or themes 
derived from the data analysis: Pedagogical Flexibility, Support and Necessary Skill, Teaching 
and Learning Quality, Framework Organization, and Comprehensiveness. The chapter 
summarized version changes to provide a homogenized description of what took place in 
updating the instructional delivery framework (Appendix F). 

Chapter 5 presents a discussion about the instructional delivery framework in the scope 
of the three research questions. The section offers a discussion of the implications for higher 
education faculty and administrators. The chapter will conclude by providing a description of 
the final version of the framework and reviews the areas of inadequacy along with a research 
agenda for continued development on the model (Reigeluth & Frick, 1999). 
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Chapter 5: Discussion and Conclusions 

The purpose of the Formative Research study was to develop an instructional delivery 
framework for distance education courses at a small private liberal arts university in western 
New York state referred to with the pseudonym Magdalene University in this study. The 
framework meets requirements from accreditation and regulatory agencies, faculty and 
administration concerns about how to design and implement quality distance courses, and 
maintenance of pedagogical flexibility for faculty. Instructional delivery items were well- 
researched, evidence-based, and institutionally sound to provide adequate guidance to 
experienced and inexperienced distance education faculty. 

The findings of the study will add to the body of knowledge specific to institutional and 
instructional leadership in higher education with an emphasis on distance education. The 
empirical data collected included observations, interviews, and focus group. The participants in 
this study were all full or part time members of the professoriate at Magdalene University. The 
study focused on the participants’ perceptions and attitudes toward the use of an instructional 
delivery framework designed to support the implementation of quality distance education 
courses. 

To address the gap in research and practice, this study created an instructional delivery 
framework that provided the support and necessary skills to instruct online courses while 
maintaining pedagogical flexibility to allow faculty to select tools and techniques appropriate for 
themselves and their respective courses. At the conclusion of the study, the developed 
instructional delivery framework was named CourseQ™. The moniker CourseQ™ is a 
compounding of the words course and quality referring to quality course delivery. CourseQ™ 
meets literature and research guidance suggesting that institutions develop individual quality 
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guidelines and instruments to support faculty in meeting quality expectations by providing an 
appealing, effective, and easy to use instrument. Three research questions framed the Formative 
Research study and revealed faculty members’ perceptions and attitudes regarding the appeal, 
effectiveness, and ease of use of the instructional delivery framework later named CourseQ™: 

RQ1: What are the perceptions and attitudes of faculty in a small university about how a 
proposed distance education quality course delivery framework is appropriate and appealing to 
them? 

RQ2: What are the perceptions and attitudes of faculty in a small university about how a 
proposed distance education quality course delivery framework will meet their needs for 
effectiveness in delivering quality online courses? 

RQ3: What are the perceptions and attitudes of faculty in a small university about how a 
proposed framework will meet their needs for ease of use? 

Chapter 5 presents a discussion about the CourseQ™ framework. This section begins 
with a summary of the findings aligned by the research questions of the study. A discussion of 
the CourseQ™ framework compared to the literature presented in Chapter 2 about quality issues 
related to both higher education and distance education. The chapter concludes with an analysis 
of the study documenting the limitations of the study, inadequacies of the instrument, and 
recommendations for continued research about the developed instructional delivery framework. 
Summary of Findings 

Chapter 4 presented a detailed reporting of results and narrative analysis of the research 
project developing the instructional delivery framework, later named CourseQ™, for quality in 
distance education delivery. Ten volunteer participants took part in the study and were divided 
into three study groups. The first two study groups had four participants each and the final group 
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had two members. Participants in this study were full or part time faculty members of the 
research location and from a range of academic disciplines. Participants had a variety of 
experiences including no experience, experience taking a course, experience teaching in a 
blended course, and teaching completely online courses. Members of this study applied the 
instructional delivery framework to a sample case and were observed, interviewed individually, 
and took part in a focus group about their respective versions of the instructional delivery 
framework. Responses from transcribed interactions were coded, categorized, and organized 
into concepts as described in Chapter 4 (Lichtman, 2013). 

Formative Research studies determine the impact of instructional models, theories, or 
frameworks based on three distinct areas: appeal, effectiveness, and efficiency or ease-of-use 
(Reigeluth & Frick, 1999). Participants in this study remarked that these three areas overlapped 
considerably, stating that in order for the model to appeal to them, it must be effective and 
efficient. Interview and focus group questions were organized according to the stated research 
questions. Below is a summary of findings pertaining to each of the research questions and 
aspects of Reigeluth and Frick’s (1999) Formative Research protocol. 

Appeal of CourseQ™ framework. The appeal of the CourseQ™ was noted throughout 
the data collection cycles of the study. Through data collection in three distinct study groups, it 
was determined that the appeal of the framework required that the instrument allow for flexibility 
and freedom to determine what teaching and learning techniques, tools, and inclusions amount to 
quality in an distance education course. The ability for faculty members to freely select and 
apply instructional content, items, and interactions is critical to the implementation of quality in 
distance education (Kirkwood & Price, 2008; S. A. Lei & Gupta, 2010; Mancuso, 2009; Schuck 


et al., 2008; Yair, 2008). 
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Research participants agreed that appeal of the CourseQ™ framework was attributable to 
the emphasis on pedagogy over technology. The pedagogical focus of the instrument allows 
faculty to first consider the goals of instruction, learning objectives, and teaching strategies. The 
decisions of faculty about teaching and learning strategies could then be mapped to technological 
tools and techniques that are empirically-based, contain examples from other faculty, and include 
setup guidance to ensure quality implementation of the tool. Stressing pedagogy above 
technological tools encourages faculty participation in online learning by addressing their 
overriding concern that technology not limit learning outcomes and student success (Graham & 
Jones, 2011). 

CourseQ™ provides faculty with a thorough system of choices, guidance, and 
instructions increases the appeal of the instrument and thus encourages use. The 
comprehensiveness of the instrument can cause confusion and the inclusion of hierarchical 
organization, instructions, and web-based formats promoted comfort with the study participants 
and thus increased appeal of the tool. 

Effectiveness of CourseQ™ framework. The CourseQ™ framework was analyzed 
according to the effectiveness of the instrument to design and deliver a distance education 
courses. Participant data about flexibility and freedom to select what interactions, items, 
techniques, and tools would best meet the course objectives overlapped with the data about 
appeal. The model does not prescribe tools or techniques but rather provides these as a method 
to encourage faculty choice about what is most appropriate for all stakeholders. Providing 
quality distance instruction is best accomplished when faculty members determine the best 
methods of meeting course objectives (Ching- San et al., 2009; Mustafa & Dalen, 2006; Yair, 


2008). 
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Effectiveness of the CourseQ™ framework was also attributable to the support provided 
in the instrument. Participants remarked that the support better enables them choose and 
implement the correct item, interaction, or tool. Support and skills development includes the 
inclusion of item name, description, empirically-based recommendations for use, examples of 
implementation from other faculty, and accurate setup instructions. This collection of support 
items serves to remind and increase faculty ability to implement a quality distance course. 
Participant data also showed that the instructions support faculty regardless of experience level 
with distance education. Scholars agree that supporting faculty skills necessary for online 
teaching builds familiarity with the modality and is central to effectually delivering quality 
distance courses and programs (Singleton & Session, 2011; Tabata & Johnsrud, 2008). 

The organization of the CourseQ™ instrument follows natural workflows for faculty 
participants; including partnerships with instructional design administrators. The ability for 
participants to follow the organization of the framework through a hierarchical organization of 
tables, links, and instructions increased the effectiveness of the instrument. Quality instruments 
that require faculty use should strive to provide an efficiencies and not complicate or add to 
existing work practices in a hindering manner (Jordens & Zepke, 2009; Shulman, 2007). 

Ease of Use of CourseQ™ framework. Data regarding the CourseQ™ framework was 
also analyzed and indicated that the efficiency the instructional delivery framework was attained 
through a series of revisions. Scholars agree quality systems for distance education need to be 
easy and simple for faculty to implement or institutions risk a poor adoption of the instrument 
(Jordens & Zepke, 2009; Shulman, 2007). CourseQ™ was moved from a paper-based system to 
a completely web-based format where redundancies were eliminated and organizational aids like 


color-coding was added. 
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Data from participants showed that the efficiency or ease of use of the CourseQ™ 
framework largely corresponded with areas of appeal and effectiveness. Analysis of the data 
indicated that the grouping of comprehensive and easy to access tools and techniques for 
distance instruction eliminated the need to do extensive searching for the correct item or 
interaction. The comprehensiveness of the instructional delivery framework was complimented 
with at-hand support items built into the instrument. These included descriptions, evidence- 
based recommendations for use, faculty examples of the interaction type, and accurate setup 
instructions to support implementation. 

Participant data also indicated that the hierarchical organization of the instrument was 
natural and simple to follow. This was complimented by instructions, which clarified and eased 
the use of a new quality instruction tool. The inclusion of a change and addition protocol, which 
easily allowed faculty to contribute and grow the document as deemed appropriate. 

Discussion of Research 

CourseQ™ meets several areas of relevant literature, research, and discussion about 
quality issues in higher education and specific to distance learning. Development of the 
CourseQ™ framework was guided by external and internal factors concerning higher education 
quality, assessment matters, faculty and administration concerns about quality in distance 
learning, accreditation issues with online education, and addresses gaps that exist in other 
popular models. 

Relationship to quality in higher education. The research and development of the 
CourseQ™ framework is based on the literature of quality in higher education. The research 
shows CourseQ™ as an instrument that integrates various approaches to higher education 
quality. Data from participants showed that the framework addresses external and internal 
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factors that influence overall quality in higher education. The researched framework also meets 
the needs for ongoing assessment practices that are evolving in higher education. Collected data 
from this study also showed that CourseQ™ the focus on quality had the potential to improve 
pedagogical technique and practice. This section discusses the results of the study in the context 
of literature presented in Chapter 2 related to elements of quality in higher education. 

An integrated quality instrument. One premise in developing the CourseQ™ framework 
was a need to shift the dialogue from whether to have a quality system or not to internal 
discussions about how quality should be ensured in academic content and course delivery (Dill, 
2010; Filippakou, 2011; Pratasavitskaya & Stensaker, 2010). Harvey and Green (1993) 
classified six approaches to quality in higher education: Exception, Perfection, Fitness for 
Purpose, Driven by Mission, Value for Money, and Transformative. From these six 
classifications, the Transformative model figures prominently. Scholars suggest that modern 
institutional quality models should balance different accountability practices from a variety of 
quality models with a focus on Transformative quality model characteristics (Harvey & 

Williams, 2010; Pratasavitskaya & Stensaker, 2010; Singh, 2010). CourseQ™ blends a variety 
of approaches to quality to meet the contemporary concerns, needs, and issues surrounding 
distance learning in higher education. 

CourseQ™ meets the Transformative precepts by empowering student learning through a 
comprehensive collection of interactions and course inclusions (Harvey & Green, 1993). Data 
from participants concluded that the pedagogical priority over technology was important to the 
adoption of the instrument. Research suggests that Transformative quality models are most 
effective when the quality instruments focus on the course and learning experiences of the 
students (Jordens & Zepke, 2009). Transformative quality models also relies on interdisciplinary 
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collaborations to foster quality course delivery (Jordens & Zepke, 2009). The CourseQ™ 
framework includes a changes protocol that allows for faculty users to submit suggestions for 
changes or additions to the instrument. The openness of this change process enables a wide 
array of disciplines to make suggestions or recommend changes to the document increasing the 
validity and usefulness of the tool. The Formative Research method used in this study followed 
the notion of interdisciplinary quality practices through the creation of participant groups 
representative of different departments and divisions of the research institution. Dialogue 
between these members created salient data that led to version changes and acceptance. 

The adaptations of departmental, institutional, and mission-driven specifications in 
CourseQ™ were results of data gathered from participants in the first round of collection. The 
suggestions of participants were that the framework needed to easily link faculty users to 
department (discipline) specifics and institutional policies that link to the mission of the college 
to support their inclusion in the course. The Mission Driven model of quality is popular in the 
United States and befitting smaller institutions with strong missions and assessment practices 
(Harvey & Green, 1993). 

Data gathered from participants about CourseQ™ showed the model using tenets of the 
Fitness for Purpose quality approach. Fitness for Purpose quality approaches ensure the 
instruction of the courses are aligned to the purpose of the organization (Harvey & Green, 1993; 
Sarrico et ah, 2010). Harvey and Green (1993) recommended that institutions using aspects of 
Fitness for Purpose continually monitor the environment to ensure that the purpose is aligned to 
the external environment. Data from participants indicated that the model allowed them to create 
instruction that is better aligned established objectives. Participant suggestions to include 
departmental and institutional specifics into the CourseQ™ framework also facilitated easy 
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reference and review of specific purposes needed in the courses. Finally, the change protocol 
element enables faculty, who are closest to changes in the environment and requirements within 
their disciplines, to encourage an evidenced-based modification to the quality instrument to 
ensure compliance with changes in the external environment and purpose for instruction. 

Addressing influencing factors in educational quality. The instructional delivery 
framework resulting from this study addresses external factors that impact quality practices at 
institutions of higher education. One important factor in the U.S. is the issue of rising costs 
(Bandyopadhyay & Lichtman, 2007; Daunoriene, 2011; Ewell, 2009; Harvey & Green, 1993; 
Hersh, 2007; Z. Lei, 2009). Data gathered from participants indicate that CourseQ™ creates an 
easy to use and effective framework for the creation and delivery of distance courses. 

Participants remarked throughout data collection that framework was straightforward. The 
collection of instructional tools and techniques for distance learning coupled with detailed 
recommendations, examples, and instructions provided a single source for efficient and effective 
implementation of online learning. Participants remarked that such an instrument eliminates the 
need to excessively search and seek out assistance thus increasing their ability to design and 
deliver a course. As an instrument of cost reduction, CourseQ™ reduces cumbersome workloads 
on faculty and increases the ability of the institution to more efficiently use human resources thus 
realizing more effective production costs for distance courses. It can be assumed that 
efficiencies in instructional workloads better enable the institution to offer more courses, sections 
of courses, and programs when instructional development time is reduced through a simple, 
efficient quality model as produced in this research. 

Increasing international growth of colleges and universities is a second external factor 
influencing quality in education (Baird, 2009; Daniel et al., 2006; Daunoriene, 2011; Singh, 
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2010; Southerland et al., 2007). Brink (2010) informed that global higher education rankings 
consist of academic quality and student experience in courses, which adds pressure to account 
for quality in the educational process. As such, becoming or remaining competitive in the global 
higher education marketplace requires establishing systems that encourage, monitor, and report 
on quality issues related to education (Daunoriene, 2011; Harvey & Green, 1993; Hersh, 2007). 
CourseQ™ evolved through the collection of information from study participants to include an 
array of interactions that promote quality by providing faculty with detailed recommendations 
for use, examples from other faculty, and accurate setup instructions to increase alignment to 
quality guidelines. Institutional competiveness is increased, then, by more accurately assuring 
potential students and stakeholders that quality is built into the delivery of distance courses 
available globally. 

CourseQ™ also addresses the internal factors presented by faculty concerns about 
educational quality. Faculty often remain at a distance from conversations and practices of 
academic quality despite their importance in establishing and maintaining the same (Houston, 
2010). Raban (2007) suggested that a culture of auditing academics inhibits their willingness to 
participate in quality processes. CourseQ™ creates a culture of participation by offering 
extensive options for instructing distance courses. Participant data that led to the inclusion of 
examples from other faculty for possible course interactions provides a repository of best 
practices contributed by members of the professoriate creating collaborations and 
communication. The addition of the change protocol taken from data furthers the culture of 
quality by providing faculty a direct voice to evolve the CourseQ™ framework as new ideas, 
techniques, and tools arise from use. By involving faculty in the research and ongoing use of the 
CourseQ™ model, the instrument promotes ongoing trust of the content and usage in online 
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courses. Raban (2007) described trust from faculty as a significant factor influencing the role of 
faculty in educational quality. 

Institutional growth and change is another internal factor that influences quality practices. 
Changes in higher education arise from the implementation of quality assurance processes 
(Houston, 2010). Quality programs that account for failure and build in learning opportunities 
about those failures allow institutions to grow and evolve (Francis, 2010). Some scholars 
recommend that quality be an internal concern that integrates administration and faculty 
together, thus improving not only educational quality, but also the whole organization (Sarrico et 
al., 2010; Schulte, 2010). CourseQ™ was developed through a collaboration amongst faculty 
participants and an administrator-researcher to ensure that the model was appropriate for use 
throughout the institution and not a redundant or burdensome instrument within the organization. 
The framework includes detailed recommendations and instructions for quality implementation 
and delivery of an online course without overriding the experience and expertise of faculty. 
Rather, the model provides ready resources to easily support faculty working in a distance 
course. CourseQ™ integrates institutional and departmental specifics that are unique to the 
research institution but could be easily replaced by specifications from other institutions in an 
implementation. 

Supporting effective assessment practices. Because of the external pressures to make 
transparent the quality practices of institutions of higher education, the misinterpretation or 
misapplication of assessment results, and the inadequate institutional focus on institutional 
assessment some scholars suggest the creation of more flexible, efficient, proactive, and 
embedded approaches to assessment practice within the college or university that provide for 
pedagogical flexibility and faculty experience (Banta, 2010; Bers, 2008; Harvey & Williams, 
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2010; Jordens & Zepke, 2009; Juceviciene, 2009; Kristensen, 2010; Pratasavitskaya & 

Stensaker, 2010; Sarrico et ak, 2010; Schuck et ak, 2008; Shulman, 2007). Data collected 
throughout the research cycles indicated that CourseQ™ is an efficient model of quality that 
provided for pedagogical flexibility and faculty expertise. The data suggests that the framework 
also serves as a repository of best practices and existing courses that make recall for assessment 
more practical and efficient. 

To some degree, traditional assessment practices and popular external oversight requests 
fail to address the quality of teaching and learning at the course level in a proactive manner 
(Pratasavitskaya & Stensaker, 2010). Institutions should make certain that contemporary 
assessment activities include pedagogy in assessment more effectively (Bers, 2008; Ziliukas & 
Katiliute, 2008). Participant remarks collected indicated that CourseQ™ effectively places 
pedagogical practice ahead of technology and provides enough support to effectively teach 
students in a distance course. Data also implied for faculty unsure about a teaching technique 
could follow the framework and create a course that met quality standards, which would aid 
assessment activities. 

Promoting quality instruction. Banta (2010) studied 146 institutions and determined that 
a focus on quality processes improved pedagogical techniques and faculty development, and 
curricula. Data collected in this study concurred with Banta’ s findings. Participants remarked 
that CourseQ™ provided an extensive collection of best practices that promoted quality teaching 
in online courses. Data suggested that CourseQ™ could be a tool for faculty development either 
as a just-in-time learning or through more formal development sessions especially for faculty 
unfamiliar with the modality or techniques. Ascough (201 1) suggested that the inclusion of 
these factors into an instructional design model is shown to improve faculty adoption of quality 
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teaching techniques and improvements in teaching and learning excellence. Data collected from 
participants indicated the appeal and effectiveness of the model is the focus on pedagogy 
agreeing with Ascough’s premise. 

An instructional quality model should not seek to limit faculty into teaching with specific 
techniques, styles, or practices but instead leverage the collaborations of instructional personnel 
to ensure that instructional techniques are well researched and evidence based (Oermann, 2007 ; 
Pratasavitskaya & Stensaker, 2010; Yair, 2008). The ability for faculty to make pedagogical 
decisions about course design and delivery is directly linked to student success and course 
satisfaction (Mancuso, 2009; McLawhon & Cutright, 2012; Selwyn, 2011). Data collected from 
participants in this study imply that CourseQ™ provides a flexible but standardizing model for 
distance instruction that does not limit but rather empowers faculty by providing choice and 
empirically based recommendations for use. Scholars agree that exemplary faculty would reject 
a rigid instructional design model as they do not feel constrained or reverent to the such 
structures (Kirkwood & Price, 2008; Yair, 2008). The CourseQ™ framework was repeatedly 
commented upon as not rigid or prescriptive but rather provided a platform for faculty to 
experiment and use their expertise in the instruction. 

Relationship to quality in distance education. The research and development of the 
CourseQ™ framework is based on the literature of quality in distance education. The research 
shows CourseQ™ as an instrument addresses faculty concerns about support for online 
instruction and pedagogy. The data collected also indicated the developed quality framework 
addresses administrative concerns about faculty adoption of distance courses, budgeting and 
resources, and quality of online learning. Research in this study indicates that the CourseQ™ 
framework meets specific demands for accreditation distance education. The current and most 
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widely used quality models for distance education, the Sloan-C Five Pillars of Quality in 
Distance Learning and Quality Matters, address different levels of quality for distance education, 
but do not address the delivery of instruction in online courses (Battin-Little, 2009; Bourne et al., 
2005; Kee Meng & Mayadas, 2010; Pollacia & McCallister, 2009; Westerfelt, 2011). 

CourseQ™ addresses gaps presented in literature related to those models offering institutions 
additional ability to ensure quality in distance education. As a collective of instructional design 
elements, tools, and techniques, the data suggests CourseQ™ promotes teaching quality in 
distance education. This section discusses how the developed instructional delivery framework 
increases the ability of institutions to offer quality distance courses and programs that meet 
demands common to online learning and are not adequately addressed by existing quality 
instruments. 

Addressing faculty concerns. A reason faculty struggle to adopt online instruction is 
reduced levels of support and skills development amongst faculty members (Graham & Jones, 
2011; LaPrade et al., 2011; Lee et al., 2010; Singleton & Session, 2011). A lack of familiarity 
amongst faculty with the necessary computing technology, poor past experiences with teaching 
in the modality, and anxiety about using computers can serve as barriers to quality distance 
education (Singleton & Session, 2011; Tabata & Johnsrud, 2008). The CourseQ™ framework 
evolved from providing a basic level of support to an extensive support system for faculty users. 
Data gathered in the research provided changes that met with faculty desire and were vetted 
through research. The developed framework contains interaction descriptions, empirically based 
recommendations for use, examples of how other faculty use the interaction, and detailed setup 
instructions. This foundation provides just-in-time support and training for faculty teaching at a 
distance. The CourseQ™ framework also provides a basis for ongoing institutional training and 
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collaboration around teaching and learning techniques for distance education. Research suggests 
that faculty members familiar with technology and believe that it is critical to the profession of 
teaching are more likely to effectively participate in distance education (Tabata & Johnsrud, 
2008). Data from the research indicated that faculty without any or much experience with online 
instruction could easily apply this model and successfully implement the course. Additional data 
on this point suggested that as faculty gain experience reliance on the framework could be 
lessened over time or used when questions or new ideas arise. Participant data about this support 
was resoundingly positive and was the basis for data collected as it applied to the appeal, 
effectiveness, and efficiency of the CourseQ™ model. 

Another aspect of support is the accounting for time invested in creating, migrating, and 
instructing online course. Both faculty and administrators understand that time investment for 
distance education is front-loaded in course creation and reduces over the life of a course 
provided there are no significant revisions to the course (Bolliger & Wasilik, 2009). CourseQ™ 
creates an efficient single resource for a collection of tools and techniques and associates these 
with evidence-based recommendations, examples of use, and detailed setup instructions to 
reduce development and creation time for faculty. Data collected suggests that the CourseQ™ 
framework makes searching for tools, techniques, setup instructions, and human resources more 
effective and efficient by placing much of that information in one place. 

Faculty express concern that distance education makes standardizing instruction simpler 
and risks the pedagogical variety important to higher education (Huett et al., 2008; Peinovich, 
2008). The CourseQ™ model was careful to not prescribe a set of tools or techniques but rather 
offer choices and recommendations for faculty to use within their courses. Data showed that the 
model encouraged freedom and flexibility for faculty to deliver the course in a way that best suits 
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their needs or objectives along with support and help for making those interactions more 
effective. Participants remarked that the comprehensiveness of this framework provided 
inspiration for trying new tools or techniques and a failsafe to correct activity if the 
implementation goes wrong during the course. 

Research shows that faculty express concern that distance learning does not provide the 
same levels of interaction and is more prone to creating isolation amongst learners that face to 
face classes (Huett et al., 2008; S. A. Lei & Gupta, 2010; Peinovich, 2008; Tanner et al., 2009b). 
Participants commented throughout the data collection cycles that distance learning can provide 
a more robust learning experience but still lacks the interactivity of face-to-face classes. The 
inclusion of synchronous interaction addressed these faculty concerns. Contrary research 
suggests that online classes can provide a greater level of interaction but agree that this is not the 
same as face-to-face collaborations (Cook et al., 2009; Orellana, 2006; Singleton & Session, 
2011). The CourseQ™ model provides recommendations and supporting context to assist 
faculty with creating an interactive course that meets expectations and accreditation guidelines. 
Interactivity is sometimes linked to class size and the ability of faculty to interact with a large 
class size (Cook et al., 2009; Orellana, 2006; Singleton & Session, 2011). CourseQ™ did not 
address the topic of class size leaving this to institutional and departmental policy. Orellana 
(2006) addressed this research and concluded that not enough evidence supports whether or not 
class size has an impact on the level of interactivity. 

The growth of distance learning creates faculty questions and concerns about the best 
methods for instructing an online course (Kupczynski et al., 2012). Faculty concerns about the 
quality teaching of distance education courses evolve from concerns due to a lack of familiarity 
with the modality, to concerns about skills and abilities in teaching in this environment, to 
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concerns about the success of students (Graham & Jones, 2011; Hae-Deok et al., 2011). These 
concerns are abated and faculty use is encouraged when a widely supported technological 
infrastructure that includes support for technical skills development and flexible course 
technologies is in place (Al-Salman, 2011; Chien et al., 2004; Graham & Jones, 2011; Hae-Deok 
et al., 2011; Lee et al., 2010; Orr et al., 2009). CourseQ™ follows this research by providing a 
large collection of tools and techniques combined with detailed support including 
recommendations, examples, and instructions for setup. Creating a system that enables faculty to 
create on their own or more effectively partner with others to create a course that meets quality 
expectations. 

Addressing administrative concerns. To remain competitive and relevant in an 
increasingly diverse marketplace for higher education, traditional school administrators are 
strategically planning and encouraging the growth of distance education programs (Bolliger & 
Wasilik, 2009). This expansion into online learning is tempered with an awareness that in some 
cases, faculty lack the necessary skills to effectively instruct a growing number of courses placed 
online (Gaytan, 2009; Orr et al., 2009). The results of the data demonstrate that CourseQ™ 
provides not only the support and necessary skills but also serves as platform for seeking more 
assistance. As technology and technique evolve, CourseQ™ provides a change protocol to assist 
the framework in meeting the changing demands on distance education. The expansion to online 
learning is also tempered with concern about rising costs and shrinking budgets at many 
institutions (S. A. Lei & Gupta, 2010). The developed framework has the potential to help 
balance cost areas such as faculty workloads, development costs, and streamlining delivery of 
courses. Results of the data analysis indicate that having all the materials in once simple 
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interface does help make the design and delivery more efficient by reducing the amount of 
searching and sourcing various resources and people. 

Administrators agree with faculty that quality is integral to the continuation and 
expansion of distance education and that it is possible with the use of quality frameworks that 
support development and implementation of online courses (Chien et al., 2004; Gaytan, 2009). 
Distance education, particularly, requires institutions to establish quality measures to assure the 
public that the online courses meet expectations and organizational mission (Daniel et al., 2006; 
Jung et al., 2011; Peinovich, 2008; Seok, 2007). Results of the data indicated that the empirical 
nature of the CourseQ™ framework assists the delivery of quality distance courses. As an 
instrument of quality, data show that CourseQ™ puts pedagogical quality ahead of technological 
implementation of teaching. This aspect of the model enables faculty to effectively determine 
how best to meet the objectives of the course within the context of the institutional quality 
expectations. CourseQ™ is the instrument that eases the burden of maintaining quality within 
the college. 

Relationship to distance education accreditation issues. Accreditation in distance 
education is an extension of traditional accreditation standards and expectations (Middle States 
Commission on Higher Education, 2011; Seok, 2007). External pressures exist for governing 
agencies to standardize the assessment of quality in distance education but this devalues 
institutions be homogenizing instruction in this modality (Sarrico et al., 2010). Seok (2007) 
recommended that institutions develop or adapt their own model of quality for distance 
education. CourseQ™ contains instructional elements outlined in the Best Practices for 
Electronically Offered Degree and Certificate Programs or Guidelines for the Evaluation of 
Electronically Offered Degree and Certificate Programs developed by the Western Cooperative 
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for Educational Technology or WCET and common to all regions of the Higher Learning 
Commission. This intentional design created design elements that included interactions and 
support for meeting the expectations of accreditation. Data analysis revealed that the 
organization and categories included in the CourseQ™ framework guided faculty through that 
was required to produce a course. Quality elements for this research institution were represented 
by the links to department and institutional specific policies and practices. Recommendations for 
use, use examples, and setup instructions were vetted through literature, institutional policy, and 
implementation instructions from common tool providers. 

Addressing gaps in existing models. Several individual models do exist as guidelines for 
institutions developing or needing to evaluate distance education course quality (H. Wang, 2008; 
Y. Wang & Miller, 2006). Of the models in existence, two remain popular and most widely 
used: Quality Matters, and the Sloan-C Five Pillars of Quality in Distance Learning (Battin- 
Little, 2009; Bento & White, 2010; Bourne et al., 2005; Kee Meng & Mayadas, 2010; H. Wang, 
2008; Y. Wang & Miller, 2006; Westerfelt, 2011). Quality Matters while providing an easy-to- 
follow, research-based, guide for faculty members to design their own online courses does not 
account for the interactions possible within a distance course environment or address unfamiliar 
tools (Battin-Little, 2009; Bento & White, 2010; Pollacia & McCallister, 2009). This can make it 
easier for novice distance education faculty to review the setup and design of the course but does 
not adequately address how effective the course was at delivering education to the students 
through various pedagogical methods (Battin-Little, 2009; Pollacia & McCallister, 2009). The 
Sloan-C Five Pillars of Quality Distance Learning provides institutions a guideline for the 
development of full programs to deliver online to students (Bourne et ah, 2005; H. Wang, 2008). 
The model loosely addresses the delivery of course materials in the learning effectiveness 
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domain, but it is more focused on the construction and institutional support of the course (Bourne 
et al., 2005; Clark et al., 2009; H. Wang, 2008). While effective at the institutional assessment 
level of quality, it fails to adequately address uses of distance education technology, teaching 
practices, or learning activities in online courses (Bourne et al., 2005; Kee Meng & Mayadas, 
2010; Westerfelt, 2011). 

The research data collect demonstrate that CourseQ™ closes potential gaps that exists in 
both the Quality Matters and Sloan-C Pillars. Quality Matters, while effective at addressing 
issues of design, does not address the issues of course delivery (Battin-Little, 2009; Pollacia & 
McCallister, 2009). The CourseQ™ framework provides institutionally-oriented design and 
delivery guidance as determined both by participants and researched evidence. CourseQ™ 
specifically and with detail addresses the design and delivery of the distance courses by 
providing pedagogical level guidance. It is not the intention of this research to supplant existing 
models but rather address potential gaps in the field that faculty and administrators feel are 
important to be addressed. Scholars proffer that the creation of new blended models of quality 
better equip the institution to meet changing and evolving demands from stakeholders and 
accreditors (Jordens & Zepke, 2009). 

Implications for Leadership 

The research and development of CourseQ™ has implications for leadership in higher 
education. The analysis of the research data revealed five concepts or themes: Pedagogical 
Flexibility, Teaching and Learning Quality, Support and Necessary Skills, Framework 
Organization, and Comprehensiveness. These concepts applied in various ways to the three 
research questions relating to appeal, effectiveness, and ease of use. 
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The first implication of this study to leadership is the method used and resulting 
framework. The research and methodology presented in this study demonstrates one method of 
creating a quality instrument appropriate for supporting quality distance education. Seok (2007) 
recommended that institutions develop or adapt their own models using quality benchmarks that 
fit the organization best. Through a faculty-administrative collaboration and using a Formative 
Research method, an institution can confidently provide stakeholders a transparent and rigorous 
quality model that meets stakeholder expectations, accreditation guidelines, and regulations. 

Related to the creation of a model for quality distance instruction, the research provided 
here presents a model that met faculty appeal, needs for effectiveness, and expectations of 
efficiency without forgoing a focus on pedagogical flexibility, teaching and learning quality, and 
support for teaching online. External pressure to standardize assessment and thus instruction in 
online class environment threatens the variety important to quality collegiate education (Huett et 
al., 2008; Peinovich, 2008). Scholars contend that providing a rigid and standardized model of 
instruction or quality to faculty would undermine the intention to create a quality distance 
experience for students (Schuck et al., 2008). The information presented in this study bridge the 
need for quality standards and pedagogical flexibility to ensure quality by offering a 
comprehensive set of choices supported by detailed instructions. The CourseQ™ model is not 
all encompassing since distance education is a quickly evolving field. Instead, the model 
provides for growth through and change protocol allowing users to submit evidence or 
experience-based additions or modifications to the framework. This is particularly important to 
leadership at institutions that prescribe instructional content for distance learning or limit the 
pedagogy by demonstrating that it is possible to ensure quality and compliance without 
sacrificing the flexibility and freedom central to teaching and learning quality. 
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Scholars contend that leadership decisions made regarding distance education often fail 
to consider faculty resource, available time, budget, or adequate policies addressing a different 
instructional modality (Bolliger & Wasilik, 2009; Graham & Jones, 2011; Orr et ah, 2009; 
Singleton & Session, 2011). Implications of this study for faculty and administrative leadership 
include the efficiencies discovered in using the CourseQ™ framework. Analysis revealed that 
using the framework made creating an online course easier and more convenient despite the large 
collection of possible tools and techniques included in the instrument. This could provide 
information used to make decisions or policies about course load assignments, incentives for 
instructing online, workforce planning for new distance programs, or periodic accreditation 
projects. 

The use of CourseQ™ in this study and foreseeable application in an institution provides 
additional implications for leadership in higher education. Use of the instrument should be 
encouraged amongst faculty and instructional administration. Scholars concur that 
collaborations amongst faculty and administration produce a climate that actively supports 
quality and assessment practices (Sarrico et al., 2010; Schulte, 2010). CourseQ™ as 
recommended instrument without enforcement should be encouraged through faculty 
development, collection of faculty use examples, and involvement of faculty using the model. It 
is conceivable that CourseQ™ becomes an institutionally required model but it is suggested that 
this requirement come with strong faculty involvement and agreement to ensure adoption meets 
the intended goal of delivering quality online courses. In either an encouraged or compelled use 
case CourseQ™ should be a peer-reviewed process. Faculty members using the model to deliver 
quality online courses should participate in a partner or peer-review process where delivered 
courses are examined to determine how well the class used the instrument and met the quality 
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objectives. Faculty members using the instrument can easily self-assess their course delivery to 
assist faculty, departments, and institutions more quickly meet distance education quality goals. 

As institutions increasingly use distance learning to expand enrollment, program offering, 
and global presence, CourseQ™ provides a foundation to make that growth quality oriented. 

The implications for leadership are this research provides guidance on using quality delivery of 
online courses to foster institutional growth. Research shows that quality instruction is the most 
certain method to sustain and scale growth within an institution (Peinovich, 2008). 

Limitations of the Study 

Reigeluth and Frick (1999) suggest in their research method description that Formative 
Research will not produce a framework or model that is full tested in every possible situation. 
The scholars recommend that researchers developing a theory, model, or framework should 
identify and describe inadequacies or limitations in offering the product to the general public 
(Reigeluth & Frick, 1999). Researchers should also offer a suggestions for future research and 
development of the theory, model, or framework (Reigeluth & Frick, 1999). 

This research study used ten participants from a range of disciplines and experience with 
distance education. Within the possible pool of participants, this sample was representative 
enough to offer valid and reliable data supporting the research. Sample size could have been 
larger using large study groups or additional recruitment practices. 

One area of inadequacy in the design of CourseQ™ is the lack of full semester course 
testing. The research requested that participants design a sample course and provide feedback 
through interviews and focus group meetings. The sample case (Appendix H) was a generic, 
multi-discipline course that participants in the study would recognize and have some 
understanding of how to teach the topic. 
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Formative Research studies are related to a Type 1 case study method (Reigeluth & 

Frick, 1999). This is an appropriate approach for the development of a proposed theory, model, 
or framework as was done through this study. The restriction to one institutional site is a 
limitation to the study and future studies can address this by involving multiple research sites and 
a prolonged study of the CourseQ™ framework within various institutions. 

Recommendations for Future Research 

Throughout the research suggestions for future studies arose and are provided here for 
future research in the area of higher education and distance learning. The recommendations 
relate to the limitations identified above as well as ideas generated from conducting the research. 

One limiting factor of this study was the sample size. While the sample for this study 
was representative of the potential pool of participants, it is area that can be addressed in future 
studies. It is recommended that extensive recruitment methods be used to expand the sample 
size and draw additional conclusions about an instructional delivery framework like CourseQ™. 

The research leading to the creation of CourseQ™ was intentionally focused on faculty 
members as the primary designers and instructors of distance education courses. Future research 
could investigate the perceptions and feelings of higher education administrators regarding such 
a delivery framework. Related to this recommendation, future research might also include a 
blended population of faculty and administrators to broaden the impact of CourseQ™ model in 
institutions of higher learning. 

Another limitation of the study was the lack a semester-long research study. The 
participants took part in creating a fictitious sample course and did so in a focused time frame 
shorter than a normal working time or semester course. It is recommended that further research 
be completed that follows study participants through their normal working routines while using 
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an instructional delivery framework such as CourseQ™. Similarly, future studies should 
investigate the impact of using such a model through the implementation of the course during a 
normal instructional period. This extended research should also involve students in the courses 
and include interviews and focus groups about the appeal, effectiveness, and ease of use of the 
interactions and tools used in the course designed through this framework. 

Formative Research studies are designed to be similar to single case studies and typically 
only investigate a model within an individual institution (Reigeluth & Frick, 1999). It is 
recommended that the instructional delivery framework developed in this study be researched in 
other institutions of varying size in order to increase the generalizability and reliability of the 
model. It is recommended that CourseQ™ be researched in varying types of institutions 
including public, private, and for-profit institutions of higher learning. Related to this 
recommendation, research should be conducted with different faculty employment environments 
including both union and non-union institutions. 

Distance education is subject to speculation and mixed research results about the time 
demands of course creation and delivery. Data collected in this study suggests CourseQ™ eases 
the burden of course creation by having a single resource of collected interactions and detailed 
support. A future study should be conducted that provides information about the efficiency of 
using such a model for creating a quality distance course. 

The CourseQ™ model has potential applications in institutional assessment as described 
in this chapter. It is recommended that future research study the impact of using the CourseQ™ 
framework as it relates to assessment practices such as self-studies, accreditation reviews, or 
programmatic accreditation processes. 
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A final recommendation for future research is that investigators study the impact of 
student learning related to the use of a model like CourseQ™ compared to a course that does not 
use such a model. The information from that study would add to the validity of the model’s use 
in higher education. 

Summary 

The problem was that quality evaluation processes for distance education do not account 
for the complex differences in pedagogical approaches and instructional delivery across 
disciplines or institutions (Endean et al., 2010; Forsyth et al., 2010; Picciano, 2009; Postek et al., 
2010; P. S. Smith, 2011; Westerfelt, 2011). The purpose of this Formative Research study was 
to develop an instructional delivery framework for distance education courses at a small private 
not-for-profit university in western New York state that met requirements from accreditation and 
regulatory agencies, faculty and administration concerns about how to design and implement 
quality distance courses, and maintenance of pedagogical flexibility for faculty. The research 
presented here shows the development of a proposed instructional delivery framework, 
CourseQ™, that provides a flexible, comprehensive, and quality-oriented model for the delivery 
of distance courses in a traditional university environment. 

The research conducted indicates that the developed framework enables faculty of 
various disciplines and experience with online instruction to design a course for distance delivery 
that meets institutional, accreditation, and regulatory demands while allowing for the freedom 
customary to faculty membership. CourseQ™ followed a Formative Research method that 
involved collaboration between faculty and administration to produce an appealing, effective, 
and efficient instrument that can be implemented within the institution to support the growth of 
quality distance courses and programs. 
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Appendix A: Permission to Conduct Research 

[Permission and Authorization Forms redacted and on file with researcher] 
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Appendix B: Letter of Invitation 

E-MAIL SUBJECT LINE: Seeking Research Participants for Formative Research study 

Dear Colleague, 

You were sent this message because you are on the faculty e-mail list at [College Name 
Redacted], Mike Berta, [Title Redacted], is a graduate student in the Doctorate of Education - 
Curriculum and Instruction Program at The University of Phoenix. As part of his degree 
requirements, he is conducting a Formative Research study at [College Name Redacted] entitled, 

DISTANCE EDUCATION QUALITY COURSE DELIVERY FRAMEWORK: A 
FORMATIVE RESEARCH STUDY . 

The goal of his study is to gather opinions, facts, and observations from members of the campus 
community for the subsequent development and implementation of an instructional design theory 
for distance education courses that incorporates pedagogies across various disciplines. Data for 
this study will be collected using observations, semi-structured interviews, and focus groups. He 
is looking for voluntary participants to discuss the experiences with him. 

If you agree to participate in this study, your participation will involve the use of a drafted 
instructional design framework in application to a hypothetical case while being observed by the 
researcher, individual interviews with the researcher in person or via video conference 
technology, focus groups with other study participants. Data will be recorded via audio recording 
device and transcribed into a secure electronic file. Your participation should not exceed five (5) 
hours throughout the scope of the study. 

The researcher may terminate your participation in this student if you do not attend a study- 
related appointment without rescheduling or do not participate in one element of the data 
collection (observation, interview, and focus group). You can decide to be a part of this study or 
not. Once you start, you can withdraw from the study at any time without any penalty or loss of 
benefits. The results of the research study may be published but your identity will remain 
confidential and your name will not be made known to any outside party. He will be asking you 
questions about your experience with reviewing and applying a drafted framework to a 
hypothetical case. 

The observation phase will take approximately 45 minutes to 1 hour and any discussion will be 
audiotape-recorded The interview will last about 45 minutes to 1 hour and all the questions as 
well as your answers, and any discussion, will be audiotape-recorded. You can choose not to 
answer any of the questions, and you may also choose to end the interview at any time. After the 
interview, the recording will be transcribed without any identifying information, meaning there 
will be no way to link you to the study. The recording will then be deleted after the interview is 
transcribed. The transcribed documents will be shredded after a three-year period. The focus 
group sessions will last approximately 45 minutes to 1 hour any discussion, will be audiotape- 
recorded. You can choose not to answer any of the questions, and you may also choose to leave 
the focus group session at any time. 
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Your decision to participate or not participate in the study will in no way affect your current 
employment status at [College Name Redacted]. Your identity will remain confidential and will 
not be shared with anyone at [College Name Redacted] or at the University of Phoenix. The 
transcriptions of these interviews will not be released to anyone at any time. A summary of the 
findings will appear in my dissertation, and the instructional design recommendation will be 
made to the campus community. To maintain confidentiality, the data will be in summary form 
and will never contain any identifying information. 

If you are interested in participating in the study, please e-mail Mike at: [Contact Information 
Redacted] 

If you have any questions about your rights as a research participant or if you would like 
additional information to assist you in reaching a decision about participation, please feel free to 
contact the University of Phoenix IRB at [Contact Information Redacted] or [Institutional 
Information Redacted] . If you would like additional information about the study, you may 
contact Mike's dissertation committee chairperson, [Contact Information Redacted] 

Sincerely, 

[Contact Information Redacted] 


The content of this email message has been approved by the Human Subjects Research Review 
Board (IRB) at [Institution Name Redacted] and the IRB at The University of Phoenix. 
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Appendix C: Informed Consent Form 

ix® 

INFORMED CONSENT: PARTICIPANTS 18 YEARS OF AGE AND OLDER 

Dear Study Participant, 

My name is Michael R. Berta and I am a student at the University of Phoenix working on a 
Doctor of Education degree. I am doing a research study entitled Distance education quality 
course delivery framework: A formative research study. The purpose of this Formative Research 
study is to develop an instructional delivery framework for distance education courses at a small 
private liberal arts university in western New York state. The framework must meet requirements 
from accreditation and regulatory agencies, faculty and administration concerns about how to 
design and implement quality distance courses, and maintenance of pedagogical flexibility for 
faculty. Instructional delivery items should be well researched and evidence-based to provide 
adequate guidance to experienced and inexperienced distance education faculty. The researcher 
will collect data using observations of study participants, semi-structured interviews, and focus 
groups. 

Your participation will involve the use of a drafted instructional design framework in application 
to a hypothetical case while being observed by the researcher, individual interviews with the 
researcher in person or via videoconference technology, focus groups with other study 
participants. Data will be recorded via audio recording device for each the observation, 
interview, and focus group phases of the research. Audio recordings will be transcribed into a 
secure electronic file and deleted upon transcription. Your participation should not exceed five 
(5) hours throughout the scope of the study. The researcher may terminate your participation in 
this student if you do not attend a study-related appointment without rescheduling or do not 
participate in one element of the data collection (observation, interview, and focus group). You 
can decide to be a part of this study or not. Once you start, you can withdraw from the study at 
any time without any penalty or loss of benefits by emailing the researcher. The results of the 
research study may be published but your identity will remain confidential and your name will 
not be made known to any outside party. Upon the request of a participant, therefore, the 
principle investigator has sole and full access to the secured and protected research documents 
and will remove and shred all data related to any participant at their request including focus 
group data that specifically identifies the participant or originates from the participant. 



In this research, there are no foreseeable risks to you except none. 



Ill 


Although there may be no direct benefit to you, a possible benefit from your being part of this 
study is the improvement of your knowledge, skill, and ability in designing and implementing 
distance education courses. 

If you have any questions about the research study, please call me at [Contact Information 
Redacted] or email me at [Contact Information Redacted] . For questions about your rights as a 
study participant, or any concerns or complaints, please contact the University of Phoenix 
Institutional Review Board via email at [Contact Information Redacted], 

As a participant in this study, you should understand the following: 

1. You may decide not to be part of this study or you may want to withdraw from the study 
at any time. If you want to withdraw, you can do so without any problems. 

2. Your identity will be kept confidential. 

3. Michael R. Berta, the researcher, has fully explained the nature of the research study and 
has answered all of your questions and concerns. 

4. Observations, interviews, and focus groups will be audio recorded you must give 
permission for the researcher, Michael R. Berta, to record the observation, interview, and 
focus group at the time of the event. You understand that the information from the 
recorded interviews will be transcribed. Once transcribed the audio will be deleted. The 
researcher will develop a way to code the data to assure that your name is protected. 

5. Data will be kept in a secure and locked area. The data will be kept for three years, and 
then destroyed. 

6. The results of this study may be published. 

“By signing this form, you agree that you understand the nature of the study, the possible risks to 
you as a participant, and how your identity will be kept confidential. When you sign this form, 
this means that you are 18 years old or older and that you give your permission to volunteer as a 
participant in the study that is described here.” 

(CHECK ONE) (| |) I accept the above terms. (I I) I do not accept the above terms. 


Signature of the interviewee 


Date 


Signature of the researcher 


Date 



178 


Appendix D: Interview Protocol 
Part I: Notes for the Interviewer 
Overview 

1 . Tape-record the interviews if permission is granted 

2. Interview in a neutral setting. 

3. Each interview lasts 45 to 60 minutes. 

Interview Methodology 

Interviews will be implemented with a customized approach allowing for an in-depth 
investigation. Follow-up questions will be used to stimulate interviewee memory. The 
interviewer will use a semi-structured question design (Part III). Interview contains: 

1. A predetermined set of 10-15 questions 

2. All predetermined questions are the same for each participant 

Designation of Interviewee: 

Location of Interview: 

Date: 

Start Time: 

Finish Time: 

Part II: Components of the Interview 
Components of the Interview 

1. Introduction (5-10 minutes) 

2. Review confidentiality and consent form. 

3. Create a relaxed environment 


4. 


Dialogue 
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Question: Have you received my introductory correspondence explaining my research and the 
format that will be used? 

Question: Have you participated in applying the quality course design framework to the case 
provided? 

Question: Are there any questions? 

Explain the purpose of the interview 

1 . The purpose of this interview is to explore factors that influence your decisions. 
During the time we have together I would like to get an understanding of your 
experiences and observations pertinent to the subject matter of the study. 

Ask permission to record interview 

2. With your authorization, I would like to tape-record our discussion to get an inclusive 
record of what is said, since the notes I take will not be as comprehensive as I will 
require. No one other than I will listen to anything you say to me. Only I will have 
access to the records. The research results will describe what you and others have said 
predominantly in summation. No responses will be ascribed to you by name. 
Recordings will be secured upon transcription along with the transcribed interview. 

3. The open-ended questions are intended to obtain your personal experience and 
perceptions. The interview time may take about 1 hour. 

4. Would you give me permission to tape the interview? 

5. Do you have any questions before we begin? 
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Part III: Interview Questions 

RQ1: What are the perceptions and attitudes of faculty in a small university about how a 
proposed distance education quality course delivery framework is appropriate and appealing to 
them ? 

1 . What specific elements of the framework appealed to you as a faculty member designing 
your own courses for distance delivery? 

2. What specific elements of the framework did not appeal to you as a faculty member 
designing your own course for distance delivery? 

3. What would you keep the same so the framework appeals to you as a faculty member? 

4. How would you change the framework so it is more appealing to you as a faculty 
member? 

RQ2: What are the perceptions and attitudes of faculty in the small university about how 
a proposed distance education quality course delivery framework will meet their needs for 
effectiveness in delivering quality online courses? 

5. How does the framework help you in designing a course for distance delivery? 

6. What specific elements of the framework helped you as a faculty member in designing 
your own courses for distance delivery? 

7. What specific elements of the framework did not help you as a faculty member in 
designing your own courses for distance delivery? 

8. What do you recommend be kept the same so the framework helps you design your own 


course for distance delivery? 
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9. What do you recommend be changed so the framework is more helpful in meeting your 
needs in designing your own course for distance delivery? 

RQ3: What are the perceptions and attitudes of faculty in a small university about how a 
proposed framework will meet their needs for ease of use? 

10. How simple or convenient was the framework to use in designing the sample case? 

11. What about the framework was confusing or difficult to use? 

12. What about the framework would you keep the same so it continues to be simple and 
convenient to use? 

13. What about the framework would you change so it becomes simpler and convenient to 
use? 
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Appendix E: Focus Group Protocol 
Part I: Notes for the Interviewer 
Overview 

1. Tape-record the focus group if permission is granted 

2. Focus group in a neutral setting. 

3. Focus group lasts 45 to 60 minutes. 

Focus Group Methodology 

Focus groups will be implemented with a customized approach allowing for an in-depth 
investigation. Follow-up questions were used to stimulate participant memory. The interviewer 
will use a semi-structured question design (Part III). Focus group interview contains a 
predetermined set of 5-10 questions. 

Location of Focus Group: 

Date: 

Start Time: 

Finish Time: 

Part II: Components of the Focus Group 
Components of the Interview 

1. Introduction (5-10 minutes) 

2. Review confidentiality and consent form. 

3. Create a relaxed environment 

4. Dialogue 

Question: Have you received my introductory correspondence explaining my research and the 


format that will be used? 
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Question: Have you participated in applying the quality course design framework to the case 
provided? 

Question: Are there any questions? 

Explain the purpose of the focus group 

1 . The purpose of this focus group is to explore factors that influence your decisions. 
During the time we have together I would like to get an understanding of your 
experiences and observations pertinent to the subject matter of the study. 

Ask permission to record interview 

2. With your authorization, I would like to tape-record our discussion to get an inclusive 
record of what is said, since the notes I take will not be as comprehensive as I will 
require. No one other than I will listen to anything you say to me. Only I will have 
access to the records. The research results will describe what you and others have said 
predominantly in summation. No responses will be ascribed to you by name. 
Recordings will be secured upon transcription along with the transcribed interview. 

3. The open-ended questions are intended to obtain your personal experience and 
perceptions. The focus group may take about 1 hour. 

4. Would you give me permission to tape the focus group? 

5. Do you have any questions before we begin? 
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Part III: Focus Group Questions 

RQ1: What are the perceptions and attitudes of faculty in a small university about how a 
proposed distance education quality course delivery framework is appropriate and appealing to 
them ? 

1 . What aspects of the process of using the framework did not appeal to you, as faculty 
members, designing your own distance courses? 

2. What about the framework appealed to you, as faculty members, designing your own 
distance courses? 

RQ2: What are the perceptions and attitudes of faculty in the small university about how 
a proposed distance education quality course delivery framework will meet their needs for 
effectiveness in delivering quality online courses? 

3. What do you recommend be changed so the framework is more helpful in meeting the 
needs of faculty in designing courses for distance delivery? 

4. What do you recommend be kept the same so the framework to help faculty members 
design courses for distance delivery? 

RQ3: What are the perceptions and attitudes of faculty in a small university about how a 
proposed framework will meet their needs for ease of use? 

5. What about the framework was confusing or difficult to use? 

6. What elements in the framework made designing the sample case simpler and less 


confusing? 



Course Design 
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Appendix F: Instructional Delivery Framework Versions 


Version 1 


Design Element 

Definition 

Recommendations for Use 

Course Welcome 

Welcome statement that provides general information 
about the course, the instructor, and the policies of the 
course. It builds a social or community aspect of the 
course which is important to student learning, 
outcomes and student satisfaction with the course 
(Cherng-Jyh & Chih-Hsiung, 2008; Lear, Isernhagen, 
LaCost, & King, 2009). 

The introduction should include a brief biography of the 
instructor, goals/objectives of the course, and important 
course policies. 

Course Instructions 

Instructions for students on where, how, and when 
students should access materials, technology, 
services, and support for the course 

To reduce student anxiety, build familiarity with the course 
environment, and support student learning an course 
instructions should include: technology requirements, 
navigation, access of course materials, course interactivity 
overviews, and review of student expectations (Carruth, 
Broussard, Waldmeier, Gauthier, & Mixon, 2010). 

Learning Objectives 

Specific outcome statements about what students will 
exhibit after the course 

Learning objectives should be stated clearly and 
specifically so that students can develop a strategy for 
learning throughout the course. Objectives that are clear 
and focused on student outcome enable students to check 
their performance and activity against the expected course 
outcomes. (Ying, Huamao, Ronghuai, Yanhua, & Jingjing, 
2008). 



Objectives should be listed on the course syllabus and 
carried forward into the individual course modules that 
cover those objectives. 
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Student 

Assessment! 

Student assessment instruments and 
grading systems for the course 
appropriate for distance education with 
equal rigor to face-to-face class 

See appropriate table below 

Course Materials 

Instructional and learning materials for 
students 

See appropriate table below 

Course 

Organization 

How instructional materials, activities, 
and content are organized in the course 
site. 

Materials should be organized into chronological or topical modules where all the objects 
for that section are gathered together for easy navigation and access. 

Course 

Technology 

Technology used throughout the 
course for both students and faculty 
members 

A statement about the computing hardware, software, and connection technology 
provides students clear expectations about what is needed to be successful in the course. 

Accessibility and 
Universal Design 

Course materials, learning, and 
instructional activities are equally 
available for all students 

As required by Section 504 Subpart E of the Americans with Disabilities Act, reasonable 
accommodations will be made for students requiring an adjustment to the learning 
experience. 

Universal Design for Learning principles encourage faculty members to proactively make 
course materials available in different formats, provide students alternative means of 
expressing their knowledge, and multiple ways to engage with each other. 

http://www.cast.org/udl/ 
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Faculty-Student Interaction*! 

Instructional interactions between faculty 
member and students defined with 
description, requirements, and 
techniques outlined 

See appropriate table below 

Student-Student Interaction*! 

Instructional interactions between 
students defined with description, 
requirements, and techniques outlined 

See appropriate table below 
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Learning Activities!! 

Activities directly linked to stated 
learning objectives and appropriate for 
distance education technologies 

See appropriate table below 
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Course Materials 


Interaction Type 

Explanation or Definition 

Recommendations for Use 

Textbook 

Published reading about the course topic selected 
by the faculty member or department for the course. 

Select a course text that is appropriate for the 
course based on your experience. 

Confirm with the publisher that the text is available 
in print and electronically. 

Confirm with the publisher that the text confirms to 
ADA guidelines. 

Electronic Journal Articles 

An article that appears in a refereed scholarly 
journal that exists in the school’s library databases. 

Each journal article for the class should be linked in 
the LMS through use of permalink available in 
common article databases (R. Bley, personal 
communication, March 6, 2013). 

For articles without a permalink, PDF copies can be 
placed on electronic course reserve with the library 
circulation department (Poe & Barnett-Ellis, 2007). 

For articles without permalink, PDF copies of the 
article may be attached to an object within the 
course site. 
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Printed Journal Articles 

An article that appears in a refereed scholarly 
journal that exists in the school’s library databases. 

For print journal subscriptions, articles should be 
scanned to PDF and placed electronic course 
reserve with the library circulation department (Poe 
& Barnett-Ellis, 2007) 

Other Electronic Articles 

Electronic articles from magazines, newspapers, 
journalistic sources, websites, etc. that do not 
appear in a refereed journal or source. 

Each article for the class should be linked in the 
LMS through use of permalink or URL. 

For articles without a permalink, PDF copies can be 
placed on electronic course reserve with the library 
circulation department (Poe & Barnett-Ellis, 2007) 

For articles without a permalink, PDF copies can be 
placed on electronic course reserve with the library 
circulation department (Poe & Barnett-Ellis, 2007). 

Other Printed Articles 

Articles from printed magazines, newspapers, 
journalistic sources, websites, etc. that do not 
appear in a refereed journal or source and do not 
have an electronic source. 

Printed articles should be scanned to PDF and 
placed electronic course reserve with the library 
circulation department (Poe & Barnett-Ellis, 2007). 

Video Lecture 

A video recorded lecture provided by the instructor 
of the course. Video lectures 

Video lectures can be done in any manner that is 
comfortable to the instructor (i.e.: sitting in front of 
a camera, voiced over still or animated images, or 
green screen with visuals behind). 

Videos should be constructed to be briefer than a 
full classroom lecture and address key points 
succinctly. Supplemental content should be added 
where expansion is needed. (Hughes, 2009). 

Provide supplemental materials in areas of the 
lecture that are known to be confusing (other 
videos, audio files, websites, worksheets, etc.). 
(Hughes, 2009) 

Do not have video of professor inset in another 
window of video it distracts student attention 
(Friedland & Rojas, 2008). 

Include personal still images of the lecturer in the 
video if no video of lecturer is being provided to 
students. (Hughes, 2009). 
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Audio quality should be clear and free of 
distracting background noise. 

Video should be stored remotely to support easy 
streaming or downloading for a student (YouTube) 
and secured in a manner consistent with the faculty 
member preferences. 

Video should be linked within the LMS through use 
of a permalink or URL. 

Guest Lecture Videos 

A video recorded lecture provided by a guest or 
supplemental lecturer in the course. 

Video lectures can be done in any manner that is 
comfortable to the instructor (i.e.: sitting in front of 
a camera, voiced over still or animated images, or 
green screen with visuals behind). 

Videos should be constructed to be briefer than a 
full classroom lecture and address key points 
succinctly. Supplemental content should be added 
where expansion is needed. (Hughes, 2009). 

Videos should be broken down into small topic 
chunks, by key questions, or isolated subtopics. 
(Hughes, 2009) 

Provide supplemental materials in areas of the 
lecture that are known to be confusing (other 
videos, audio files, websites, worksheets, etc.). 
(Hughes, 2009) 

Do not have video of professor inset in another 
window of video it distracts student attention 
(Friedland & Rojas, 2008). 

Include personal still images of the lecturer in the 
video if no video of lecturer is being provided to 
students. (Hughes, 2009). 

Audio quality should be clear and free of 
distracting background noise. 
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Video should be stored remotely to support easy 
streaming or downloading for a student (YouTube) 
and secured in a manner consistent with the faculty 
member preferences. 

Video should be linked within the LMS through use 
of a permalink or URL. 

Other Video 

Video material that was created by someone other 
than the course instructor that might serve as 
supplemental or primary video for a lesson, topic, 
or issue within the course. 

Video should be linked within the LMS through use 
of a permalink or URL. 

Audio quality should be clear and free of 
distracting background noise. 

Audio-Only Lecture 

A lecture recorded using only an audio-format with 
no visuals or video component. 

Audio quality should be clear and free of 
distracting background noise. 

Audio should be constructed to be briefer than a 
full classroom lecture and address key points 
succinctly. 

Video should be stored remotely to support easy 
streaming or downloading for a student (YouTube) 
and secured in a manner consistent with the faculty 
member preferences. 

Audio should be linked within the LMS through 
use of a permalink or URL. 

Other Audio Support 

Audio lecture can be used as review materials using 
common student questions, areas of confusion, and 
the corresponding answers and clarifications. 
(Guertin, 2011). 

Audio quality should be clear and free of 
distracting background noise. 

Audio should be constructed to be briefer than a 
full classroom lecture and address key points 
succinctly. 

Video should be stored remotely to support easy 
streaming or downloading for a student (YouTube) 
and secured in a manner consistent with the faculty 
member preferences. 




192 




Audio should be linked within the LMS through 
use of a permalink or URL. 

Written Lecture 

A fully written lecture covering the course topic. 

Begin by informing students of the topic to be 
covered and provide an overview of the full lecture. 

Break lecture into clear subtopics and label them 
clearly using different font and bold technique. 

Make the text of the written document black 
without embellishing with color. Instead use 
bolding, italics, and underline to provide emphasis. 

Provide synthesis to the course readings and other 
materials to guide students to important points 
about the topic. 

Use white space to separate ideas and give the 
student permission to stop. 

Consider adding stop and think sections that ask 
students questions they should be able to answer 
and annotate. 

Guided Notes 

Guided lecture notes are documents that provide 
students the key points of the lecture or topic to aid 
them in writing complete notes. They are not a 
transcription of the lecture or other materials but 
rather a guide with placeholders for students to 
document their understanding and ensure more 
accurate coverage of the topic. (Williams, Weil, & 
Porter, 2012) 

Use fill-in-the -blanks sentences that correspond to 
the lecture and course materials where students 
must write in the key word or phrase (Williams et 
ah, 2012). 

Leave white space between typed sections for 
students to insert their own thinking and 
understandings. (Williams et al., 2012) 




193 


Other Websites 

Internet sites that provide accurate, vetted, and 

Review the website for validity, accuracy, and 


valuable information to students. 

reliability. 

Post a link and description of the site to the LMS. 

Provide students the main ideas, key areas, and 
important learnings you wish them to have while 
using the website. 

If the website requires interaction, provide students 
instructions on how to accomplish the tasks. 
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Faculty-Student Interactions 


Interaction Type 

Explanation or Definition 

Recommendations for Use 

Course Announcements 

An informative and brief written notice about 
course events. 

Set announcements to expire after the time of the 
course event. 

Link announcement directly to student 
communication preferences (email, etc.). 

Use announcements to advertise or remind about 
important course events, changes in the course, or 
happenings at the institution. 

Student eMails 

Direct electronic communication to individual or 
groups of students 


Assessment Feedback 

Instructor comment about student performance and 
work on assessment devices. This might include 
assignments, tests, quizzes, projects, or other 
graded activities in the course. 

Establish time frames that inform students about 
when to expect feedback on assessment activities in 
the course. 

Discussion Board - Faculty Involved 

Asynchronous discussion forums that engage 
student dialogue around a course topic. 

Begin course discussions with low stakes 
introductory or icebreaking questions to promote 
familiarity amongst students which will support 
more robust and active discussions (Cheung, Hew, 
& Ling Ng, 2008). 

Ask or assign topics that challenge students to think 
and respond thoughtfully. Topics should not 
provide a clear either or response unless appropriate 
to the course topic. 

Provide students directions and expectations about 
how to answer and participate in discussions. This 
might include instruction about asking questions for 
clarity and understanding (Cheung et al., 2008) 

Establish deadlines and expectations about 
participation to foster responsible and ongoing 
discussions (Cheung et al., 2008) 
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Summary Activity 

A synopsis of student learning provided back to the 

Include a summary activity in the discussion forum 


instructor or class as a record of what learning took 

or elsewhere for students to summarize the 


place during the week. 

discussions had during the week. (Cheung et al., 
2008) 
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Student-Student Interactions 


Interaction Type 

Explanation or Definition 

Recommendations for Use 

Discussion Groups 


Begin course discussions with low stakes 
introductory or icebreaking questions to promote 
familiarity amongst students which will support 
more robust and active discussions (Cheung et ah, 
2008) 

Ask or assign topics that challenge students to think 
and respond thoughtfully. Topics should not 
provide a clear either or response unless appropriate 
to the course topic. 

Provide students directions and expectations about 
how to answer and participate in discussions. This 
might include instruction about asking questions for 
clarity and understanding (Cheung et al., 2008) 

Establish deadlines and expectations about 
participation to foster responsible and ongoing 
discussions (Cheung et al., 2008) 

Learning Groups or Communities 

A subset of the full class organized around topics, 
assignments, and coursework. 

Have the team create a team charter and assignment 
plan to clarify their roles, strengths, weaknesses, 
and communication preferences. (Hunsaker, Pavett, 
& Hunsaker, 2011). 

Provide students support for planning assignments 
including feedback on the charter, assignment 
planning, and readiness feedback for executing the 
assignment (Gomez, Dezhi Wu, & Passerini, 2009) 

Have students summarize communication amongst 
the team to aid both in understanding and 
accountability (Gomez et al., 2009) 

Include a peer evaluation (Gomez et al., 2009) 
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Course Blogs 


Blogs, Weblogs, or Journals are short-form writing 
areas used to capture student thoughts, ideas, and 
reflections about course topics. 


Create a blog for the class either as a general blog 
or blog around a particular topic or activity. 

Consider using a class blog where individuals can 
submit postings to a single blog. This promotes 
peer-review, commentary, and dialogue amongst 
students (Smith, 2008) 


Provide students instructions and expectations 
about contribution including word count, format, 
and style. 

Encourage student commenting on individual posts 
by using the comment feature and posing questions, 
additional thoughts, or additional resources. 
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Learning Activities 


Interaction Type 

Explanation or Definition 

Recommendations for Use 

Learning Check Quizzes 

Quiz assessments that are scheduled at the end of 
sets of content or course materials that provide 
students an opportunity to test their understanding 
and knowledge of the topic (Johnson & Kiviniemi, 
2009; Roediger III, Agarwal, McDaniel, & 
McDermott, 2011) 

Include a low- stakes or low point value quiz for 
each topic module of the course (Roediger III et ah, 
2011). 

Use a small number of questions (10) to ensure 
student completion of the assessment (Johnson & 
Kiviniemi, 2009) 

Consider placing the quizzes in the module after the 
reading and ahead of any lecture materials to ensure 
students have the baseline of knowledge needed for 
the lecture (Tao, Fore, & Forbes, 2011) 

In constructing the quiz include feedback for 
students; congratulations for correct answers and 
additional guidance for incorrect answers that send 
students back to the test (Johnson & Kiviniemi, 
2009). 

Use multiple choice or multiple answer questions 
along with repeated attempts to help student 
achieve mastery of the materials (Roediger III et al., 
2011) 

Course Blogs 

Blogs, Weblogs, or Journals are short-form writing 
areas used to capture student thoughts, ideas, and 
reflections about course topics. 

Create a blog for the class either as a general blog 
or blog around a particular topic or activity. 

Consider using a class blog where individuals can 
submit postings to a single blog. This promotes 
peer-review, commentary, and dialogue amongst 
students (Smith, 2008) 

Provide students instructions and expectations 
about contribution including word count, format, 
and style. 

Encourage student commenting on individual posts 
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by using the comment feature and posing questions, 
additional thoughts, or additional resources. 
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Student Assessment 


Interaction Type 

Explanation or Definition 

Recommendations for Use 

Testing 

A scored group of questions that pertain to the 
course or course topic. 

To ensure stable access to test request students have 
a wired connection to the Internet, maintain only 
one LMS login session at a time, and have only one 
browser window during the test. 

Limit the number of LMS testing options (appear 
all at once, randomized questions/pools, prohibit 
backtracking, etc.) to only a few to prevent 
unexpected disconnections. 

Grade the questions ahead of time including 
feedback about correct and incorrect answers. 

Incorrect answer feedback should include a brief 
explanation or hint about where to review the topic 
of the question. 

Be aware of the multiple attempts feature and use it 
in accordance with your goals for the course. 

Formatted Written Assignments 

A written assessment device like a paper using an 
accepted formatting style (APA, MLA, etc.). 

Provide students a rubric that you will use to assess 
their performance on this assignment. 

Unformatted Written Assignments 

A written assessment device that does not conform 
to an accepted formatting style but may be common 
in the industry (i.e.: memo, executive summary, 
etc.). 

Provide students a rubric that you will use to assess 
their performance on this assignment. 
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Presentations 


An oral and visually performed demonstration of 
student knowledge about a topic. 


Establish clear parameters about the presentation in 
terms of time, audience, inclusions, exclusions, and 
standards of performance. 

Provide students a rubric that you will use to assess 
their performance on this assignment. 


If recorded, have students record on a camera, 
upload video to some video storage/streaming 
location and share the link to the video in the LMS 


If the presentation is live, use web conferencing 
software to have students join you and present. 

Create a support section in the course that gives 
students tutorials and guides about how to perform 
the technical aspects of the assignment. 
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Version 2 


Course Design 


Design Element 

Definition 

Recommendations for Use 

College Specific 
Requirement 

Department 

Requirements 

Course 

Description 

An institutionally accepted 
description of the class' topics, 
outcomes, and expectations for 
students 

The course description should be 
written into a separate item within 
the course as well as appearing in the 
syllabus. This should be taken 
directly from the institutional and/or 
department website. 

Institutional List of 
Course Descriptions 

Place Holder for 
Department Drop 
Down with Links 

Course Welcome 

Welcome statement that provides 
general information about the 
course, the instructor, and the 
policies of the course. It builds a 
social or community aspect of the 
course that is important to student 
learning, outcomes and student 
satisfaction with the course 
(Cherng-Jyh & Chih-Hsiung, 
2008; Lear, Isernhagen, LaCost, 

& King, 2009). 

The introduction should include a 
brief biography of the instructor, 
goals/objectives of the course, and 
important course policies. 



Mission and 
Purpose 

A statement regarding the mission 
and/or purpose of both the 
institution and department 

The mission and purpose should be 
written into a separate item within 
the course as well as appearing in the 
syllabus. This should be taken 
directly from the institutional and/or 
department website. 

Institutional 

Mission 

Place Holder for 
Department Drop 
Down with Links 
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Core Curriculum 
Statement 

An institutional program whereby 
students must develop skills in 
key areas according to the 
academic policy of the institution 

The Core Curriculum Statement 
should be written into a separate item 
within the course as well as 
appearing in the syllabus. This 
should be taken directly from the 
institutional and/or department 
website. 

Institutional Core 
Curriculum 


Details about which aspects of the 
course meet what core curriculum 
components should be made clear 

Course 

Instructions 

Instructions for students on 
where, how, and when students 
should access materials, 
technology, services, and support 
for the course 

To reduce student anxiety, build 
familiarity with the course 
environment, and support student 
learning a course instructions should 
include: technology requirements, 
navigation, access of course 
materials, course interactivity 
overviews, and review of student 
expectations (Carruth, Broussard, 
Waldmeier, Gauthier, & Mixon, 
2010). 



A statement about the computing 
hardware, software, and connection 
technology provides students clear 
expectations about what is needed to 
be successful in the course. 
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Course Learning 
Objectives 

Specific outcome statements 
about what students will exhibit 
after the course 

Learning objectives should be stated 
clearly and specifically so that 
students can develop a strategy for 
learning throughout the course. 
Objectives that are clear and focused 
on student outcome enable students 
to check their performance and 
activity against the expected course 
outcomes. (Ying, Huamao, 
Ronghuai, Yanhua, & Jingjing, 
2008). 

Objectives should be listed on the 
course syllabus and carried forward 
into the individual course modules 
that cover those objectives. 



Student 

Student assessment instruments 

Click for appropriate table 



Assessment! 

and grading systems for the 
course appropriate for distance 
education with equal rigor to face- 
to-face class 




Course Materials 

Instructional and learning 

Click for appropriate table 




materials for students 




Course 

Organization 

How instructional materials, 
activities, and content are 
organized in the course site. 

Materials should be organized into 
chronological or topical modules 
where all the objects for that section 
are gathered together for easy 
navigation and access. 



Accessibility and 
Universal 
Design 

Course materials, learning, and 
instructional activities are equally 
available for all students 

As required by Section 504 Subpart 
E of the Americans with Disabilities 
Act, reasonable accommodations will 
be made for students requiring an 
adjustment to the learning 

Institutional 
Statement on 
Disabilities 
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experience. 



Universal Design for Learning 
principles encourage faculty 
members to proactively make course 
materials available in different 
formats, provide students alternative 
means of expressing their 
knowledge, and multiple ways to 
engage with each other. 

http://www.cast.org/udl/ 

Faculty-Student 

Instructional interactions between 

Click for appropriate table 



Interaction*! 

faculty member and students 
defined with description, 
requirements, and techniques 
outlined 




Student-Student 

Interaction*! 

Instructional interactions between 
students defined with description, 

Click for appropriate table 




requirements, and techniques 
outlined 




Learning 

Activities directly linked to stated 

Click for appropriate table 



Activities!! 

learning objectives and 
appropriate for distance education 
technologies 




* - New York State Education Department Standard f - Middle States Commission on Higher Education Standard 

$ - National League for Nursing Accreditation Commission § - Commission on Accreditation in Physical Training Education 

Standard 

B - Teacher Education Accreditation Council 
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Course Design Home Student Assessment 


Interaction Type 


Explanation or Definition 


Recommendations for Use 


Testing 


A scored group of questions that pertain to the 
course or course topic. 


To ensure stable access to test request students have a 
wired connection to the Internet, maintain only one LMS 
login session at a time, and have only one browser 
window during the test. 


Limit the number of LMS testing options (appear all at 
once, randomized questions/pools, prohibit backtracking, 
etc.) to only a few to prevent unexpected disconnections. 


Grade the questions ahead of time including feedback 
about correct and incorrect answers. 


Incorrect answer feedback should include a brief 
explanation or hint about where to review the topic of 
the question. 


Be aware of the multiple attempts feature and use it in 
accordance with your goals for the course. 
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Formatted Written 

A written assessment device like a paper using 

Provide students a rubric that you will use to assess their 

Assignments 

an accepted formatting style (APA, MLA, etc.)- 

performance on this assignment. 



Provide students links to departmental policies regarding 
formatted written assignments 



Provide students with links to industry specific or 
programmatic accreditation guidelines for formatted 
written assignments 

Unformatted Written 

A written assessment device that does not 

Provide students a rubric that you will use to assess their 

Assignments 

conform to an accepted formatting style but 
may be common in the industry (i.e.: memo, 
executive summary, etc.). 

performance on this assignment. 











Presentations 


An oral and visually performed demonstration 
of student knowledge about a topic. 
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Establish clear parameters about the presentation in 
terms of time, audience, inclusions, exclusions, and 
standards of performance. 

Provide students a rubric that you will use to assess their 
performance on this assignment. 

If recorded, have students record on a camera, upload 
video to some video storage/streaming location and 
share the link to the video in the LMS 

If the presentation is live, use web conferencing software 
to have students join you and present. 

Create a support section in the course that gives students 
tutorials and guides about how to perform the technical 
aspects of the assignment. 
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Course Design Home 

Course Materials 

Interaction Type 

Explanation or Definition 

Recommendations for Use 

Textbook 

Published reading about the course topic selected by the 
faculty member or department for the course. 

Select a course text that is appropriate for the 
course based on your experience. 

Confirm with the publisher that the text is 
available in print and electronically. 

Confirm with the publisher that the text 
confirms to ADA guidelines. 

Electronic Journal 
Articles 

An article that appears in a refereed scholarly journal that 
exists in the school’s library databases. 

Each journal article for the class should be 
linked in the LMS through use of permalink 
available in common article databases (Bley, 
personal communication, March 6, 2013). 

For articles without a permalink, PDF copies 
can be placed on electronic course reserve with 
the library circulation department (Poe & 
Bamett-Ellis, 2007). 

For articles without permalink, PDF copies of 
the article may be attached to an object within 
the course site. 
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Printed Journal Articles 

An article that appears in a refereed scholarly journal that 
exists in the school’s library databases. 

For print journal subscriptions, articles should 
be scanned to PDF and placed electronic course 
reserve with the library circulation department 
(Poe & Bamett-Ellis, 2007) 

Other Electronic 
Articles 

Electronic articles from magazines, newspapers, 
journalistic sources, websites, etc. that do not appear in a 
refereed journal or source. 

Each article for the class should be linked in the 
LMS through use of permalink or URL. 


For articles without a permalink, PDF copies 
can be placed on electronic course reserve with 
the library circulation department (Poe & 
Barnett-Ellis, 2007) 



For articles without a permalink, PDF copies 
can be placed on electronic course reserve with 
the library circulation department (Poe & 
Barnett-Ellis, 2007). 

Other Printed Articles 

Articles from printed magazines, newspapers, journalistic 
sources, websites, etc. that do not appear in a refereed 
journal or source and do not have an electronic source. 

Printed articles should be scanned to PDF and 
placed electronic course reserve with the library 
circulation department (Poe & Barnett-Ellis, 
2007). 
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Video Lecture 


A video recorded lecture provided by the instructor of the 
course. Video lectures 

Video lectures can be done in any manner that is 
comfortable to the instructor (i.e.: sitting in front 
of a camera, voiced over still or animated 
images, or green screen with visuals behind). 


Videos should be constructed to be briefer than 
a full classroom lecture and address key points 
succinctly. Supplemental content should be 
added where expansion is needed. (Hughes, 
2009). 


Provide supplemental materials in areas of the 
lecture that are known to be confusing (other 
videos, audio files, websites, worksheets, etc.). 
(Hughes, 2009) 


Do not have video of professor inset in another 
window of video it distracts student attention 
(Friedland & Rojas, 2008). 


Include personal still images of the lecturer in 
the video if no video of lecturer is being 
provided to students. (Hughes, 2009). 


Audio quality should be clear and free of 
distracting background noise. 


Video should be stored remotely to support easy 
streaming or downloading for a student 
(YouTube) and secured in a manner consistent 
with the faculty member preferences. 


Video should be linked within the LMS through 
use of a permalink or URL. 
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Guest Lecture Videos 


A video recorded lecture provided by a guest or 
supplemental lecturer in the course. 


Video lectures can be done in any manner that is 
comfortable to the instructor (i.e.: sitting in front 
of a camera, voiced over still or animated 
images, or green screen with visuals behind). 

Videos should be constructed to be briefer than 
a full classroom lecture and address key points 
succinctly. Supplemental content should be 
added where expansion is needed. (Hughes, 

2009). 

Videos should be broken down into small topic 
chunks, by key questions, or isolated subtopics. 

(Hughes, 2009) 

Provide supplemental materials in areas of the 
lecture that are known to be confusing (other 
videos, audio files, websites, worksheets, etc.). 
(Hughes, 2009) 

Do not have video of professor inset in another 
window of video it distracts student attention 

(Friedland & Rojas, 2008). 

Include personal still images of the lecturer in 
the video if no video of lecturer is being 

provided to students. (Hughes, 2009). 

Audio quality should be clear and free of 
distracting background noise. 


Video should be stored remotely to support easy 
streaming or downloading for a student 
(YouTube) and secured in a manner consistent 
with the faculty member preferences. 

Video should be linked within the LMS through 
use of a permalink or URL. 
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Other Video 

Video material that was created by someone other than the 
course instructor that might serve as supplemental or 
primary video for a lesson, topic, or issue within the 
course. 

Video should be linked within the LMS through 
use of a permalink or URL. 

Audio quality should be clear and free of 
distracting background noise. 

Audio-Only Lecture 

A lecture recorded using only an audio-format with no 
visuals or video component. 

Audio quality should be clear and free of 
distracting background noise. 

Audio should be constructed to be briefer than a 
full classroom lecture and address key points 
succinctly. 

Video should be stored remotely to support easy 
streaming or downloading for a student 
(YouTube) and secured in a manner consistent 
with the faculty member preferences. 

Audio should be linked within the LMS through 
use of a permalink or URL. 

Other Audio Support 

Audio lecture can be used as review materials using 
common student questions, areas of confusion, and the 
corresponding answers and clarifications. (Guertin, 2011). 

Audio quality should be clear and free of 
distracting background noise. 

Audio should be constructed to be briefer than a 
full classroom lecture and address key points 
succinctly. 

Video should be stored remotely to support easy 
streaming or downloading for a student 
(YouTube) and secured in a manner consistent 
with the faculty member preferences. 
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Audio should be linked within the LMS through 
use of a permalink or URL. 

Written Lecture A fully written lecture covering the course topic. Begin by informing students of the topic to be 

covered and provide an overview of the full 

lecture. 

Break lecture into clear subtopics and label 
them clearly using different font and bold 

technique. 

Make the text of the written document black 
without embellishing with color. Instead use 
bolding, italics, and underline to provide 
emphasis. 

Provide synthesis to the course readings and 
other materials to guide students to important 
points about the topic. 

Use white space to separate ideas and give the 
student permission to stop. 

Consider adding stop and think sections that ask 
students questions they should be able to answer 

and annotate. 

Guided Notes Guided lecture notes are documents that provide students Use fill-in-the -blanks sentences that correspond 

the key points of the lecture or topic to aid them in writing to the lecture and course materials where 
complete notes. They are not a transcription of the lecture students must write in the key word or phrase 
or other materials but rather a guide with placeholders for (Williams et al., 2012). 

students to document their understanding and ensure more Leave white space between typed sections for 
accurate coverage of the topic. (Williams, Weil, & Porter, students to insert their own thinking and 

2012) understandings. (Williams et al., 2012) 










Other Websites 


Internet sites that provide accurate, vetted, and valuable 
information to students. 
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Course Design Home 

Faculty-Student Interactions 

Interaction Type 

Explanation or Definition 

Recommendations for Use 

Course Announcements 

An informative and brief written notice 
about course events. 

Set announcements to expire after the time of the 
course event. 



Link announcement directly to student 
communication preferences (email, etc.). 



Use announcements to advertise or remind about 
important course events, changes in the course, or 
happenings at the institution. 

Synchronous Electronic 
Communication 

Any form of live interaction between 
faculty members and students via audio, 
video, or text (chat). 

Synchronous communication via text of chat function 
either within the LMS or other systems aids students 
in mutually constructing knowledge, negotiation, 
support, group facilitation, and group processing of 
common course tasks (Maushak & Ou, 2007) 



Live video or audio communication via any usable 
system can result in improved faculty-student 
connections, reduced communication barriers 
possible in asynchronous discussion, and faster 
corrections of student work (Huang & Hsiao, 2012). 
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Establish guidelines and expectations about what 
form of synchronous communication will be used in 
the course, provide communication and behavior 
expectations, create a schedule that is convenient for 
the class or targeted learning group, and schedule 
low-stakes introductions to the technology being used 
(Huang & Hsiao, 2012) 

Student eMails 

Direct electronic communication to 
individual or groups of students 

The email students function from within the LMS 
provides a record of the email directly to the faculty 
mailbox. This includes emails to individuals, groups, 
or all users. 

Assessment Feedback 

Instructor comment about student 
performance and work on assessment 
devices. This might include assignments, 
tests, quizzes, projects, or other graded 
activities in the course. 

Establish time frames that inform students about 
when to expect feedback on assessment activities in 
the course. 

Discussion Board - Faculty 
Involved 

Asynchronous discussion forums that 
engage student dialogue around a course 
topic. 

Begin course discussions with low stakes 
introductory or icebreaking questions to promote 
familiarity amongst students that will support more 
robust and active discussions (Cheung, Hew, & Ling 
Ng, 2008). 

Ask or assign topics that challenge students to think 
and respond thoughtfully. Topics should not provide 
a clear either or response unless appropriate to the 
course topic. 
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Provide students directions and expectations about 
how to answer and participate in discussions. This 
might include instruction about asking questions for 
clarity and understanding (Cheung et al., 2008) 

Establish deadlines and expectations about 
participation to foster responsible and ongoing 
discussions (Cheung et al., 2008) 

Summary Activity 

A synopsis of student learning provided 
back to the instructor or class as a record 
of what learning took place during the 
week. 

Include a summary activity in the discussion forum 
or elsewhere for students to summarize the 
discussions had during the week. (Cheung et al., 
2008) 
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Course Design Home 


Student-Student Interactions 


Interaction Type 

Explanation or Definition 

Recommendations for Use 

Discussion Groups 

Asynchronous, student-student, 
discussion forums for specific sets of 
students usually assigned by the faculty 
member of the course 

Begin course discussions with low stakes introductory or 
icebreaking questions to promote familiarity amongst 
students which will support more robust and active 
discussions (Cheung et al., 2008) 

Ask or assign topics that challenge students to think and 
respond thoughtfully. Topics should not provide a clear 
either or response unless appropriate to the course topic. 

Provide students directions and expectations about how 
to answer and participate in discussions. This might 
include instruction about asking questions for clarity and 
understanding (Cheung et al., 2008) 

Establish deadlines and expectations about participation 
to foster responsible and ongoing discussions (Cheung et 
al., 2008) 

Learning Groups or 
Communities 

A subset of the full class organized 
around topics, assignments, and 
coursework. 

Have the team create a team charter and assignment plan 
to clarify their roles, strengths, weaknesses, and 
communication preferences. (Hunsaker, Pavett, & 
Hunsaker, 2011). 


220 




Provide students support for planning assignments 
including feedback on the charter, assignment planning, 
and readiness feedback for executing the assignment 
(Gomez, Dezhi Wu, & Passerini, 2009) 

Have students summarize communication amongst the 
team to aid both in understanding and accountability 
(Gomez et al., 2009) 

Include a peer evaluation (Gomez et al., 2009) 

Course Blogs 

Blogs, Weblogs, or Journals are short- 
form writing areas used to capture 
student thoughts, ideas, and reflections 
about course topics. 

Create a blog for the class either as a general blog or 
blog around a particular topic or activity. 

Consider using a class blog where individuals can 
submit postings to a single blog. This promotes peer- 
review, commentary, and dialogue amongst students 
(Smith, 2008) 

Provide students instructions and expectations about 
contribution including word count, format, and style. 

Encourage student commenting on individual posts by 
using the comment feature and posing questions, 
additional thoughts, or additional resources. 
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Course Design Home 


Learning Activities 


Interaction Type 


Explanation or Definition 


Recommendations for Use 


Learning Check Quizzes 


Quiz assessments that are scheduled at the end 
of sets of content or course materials that 
provide students an opportunity to test their 
understanding and knowledge of the topic 
(Johnson & Kiviniemi, 2009; Roediger III, 
Agarwal, McDaniel, & McDermott, 2011) 


Include a low-stakes or low point value quiz for each 
topic module of the course (Roediger III et al., 2011). 


Use a small number of questions (10) to ensure 
student completion of the assessment (Johnson & 
Kiviniemi, 2009) 

Consider placing the quizzes in the module after the 
reading and ahead of any lecture materials to ensure 
students have the baseline of knowledge needed for 
the lecture (Tao, Fore, & Forbes, 2011) 


In constructing the quiz include feedback for 
students; congratulations for correct answers and 
additional guidance for incorrect answers that send 
students back to the test (Johnson & Kiviniemi, 
2009). 


Use multiple choice or multiple answer questions 
along with repeated attempts to help student achieve 
mastery of the materials (Roediger III et al., 201 1) 
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Course Blogs 


Blogs, Weblogs, or Journals are short-form 
writing areas used to capture student thoughts, 
ideas, and reflections about course topics. 


Create a blog for the class either as a general blog or 
blog around a particular topic or activity. 


Consider using a class blog where individuals can 
submit postings to a single blog. This promotes peer- 
review, commentary, and dialogue amongst students 
(Smith, 2008) 


Provide students instructions and expectations about 
contribution including word count, format, and style. 


Encourage student commenting on individual posts 
by using the comment feature and posing questions, 
additional thoughts, or additional resources. 
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Version 3 

The following are screenshots of the web-interface for Version 3. Some of the screen is not shown in these images. 


Welcome to the Instructional Delivery Framework for onlme/distance courses. The goal of this tool is to help 
you successful design and implement a course for delivery online through our Learning Management System 
(Blackboard). Below are some steps that will help you use this framework. 

1. Click the ‘dick Here to Begin' link below when you have read these instructions. 

2. The Course Design page is our Home page. All pages contain a link back to this page 

3. With your course description and objectives, review the Course Design page and carefully consider what 
aspects of the framework you will include. 

4. As you work through the framework use the hyperlinks to access more information about that item. 

5. When you get to the interactions and activities in the course, you will find links to other pages which 
provide in-depth guidance about different interaction types and uses. 

6. Each of these interaction/activity pages contain a senes of items for you to consider. Based on your 
experience and information from the framework carefully make a selection of what items make sense to you 
in your course. 

7. If you feel as though something is missing, there is a link on each page to submit a new item, change, or 
make a suggestion 


Click Here to Begin 
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Course Design 

Design Element 

Definition 

Recommendations for Use 

College Specific Requirement 

Department Requirements 

Course Description 

An institutionally accepted description of the class' topics, 
outcomes, and expectations for students 

The course description should be written into a spearate 
item within the course as well as appearing in the 
syllabus. This should be taken directly from the 
institutional and/or department website. 

Institutional List of Course 
DescriDtions 

Place Holder for Department 
Drop Down with Links 

Course Welcome 

Welcome statement that provides general information 
about the course, the instructor, and the policies of the 
course. It builds a social or community aspect of the course 
which is important to student learning, outcomes and 
student satisfaction with the course (Cherng-Jyh & 
Chih-Hsiung, 2008; Lear, Isernhagen, LaCost, & King, 2009). 

The introduction should include a brief biography of the 
instructor, goals/objectives of the course, and important 
course policies. 



Mission and Purpose 

A statement regarding the mission and/or purpose of both 
the institution and department 

The mission and purpose should be written into a 
spearate item within the course as well as appearing in 
the syllabus. This should be taken directly from the 
institutional and/or department website. 

Institutional Mission 

Place Holder for Department 
Drop Down with Links 

Core Curriculum 
Statement 

An institutional program whereby students must develop 
skills in key areas according to the academic policy of the 
institution 

The Core Curriculum Statement should be written into a 
spearate item within the course as well as appearing in 
the syllabus. This should be taken directly from the 
institutional and/or department website. 

Details about which aspects of the course meet what 
core curriculum components should be made clear 

Institutional Core Curriculum 


Course Instructions 

Instructions for students on where, how, and when students 
should access materials, technology, services, and support 
for the course 

To reduce student anxiety, build familiarity with the 
course environment, and support student learning an 
course instructions should include: technology 
requirements, navigation, access of course materials, 
course interactivity overviews, and review of student 
expectations (Carruth, Broussard, Waldmeier, Gauthier, & 
Mixon, 2010). 

A statement about the computing hardware, software, 
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Faculty-Student Interactions 

Interaction Type 

explanation or Definition 

Recommendations for Use 

Faculty Use Examples 

Setup Instructions 

Course Announcements 

An informative and brief written notice 
about course events. 

Set announcements to expire after the time of the course event. 

Placeholder for Faculty Use 
Examples 

Creating Arnouncments 

Link announcement directly to student communication 
preferences (email, etc). 

Use announcements to advertise or remind about important 
roursc events, changes in the course, or happenings at the 
institution. 


Synchronous Electronic 
Communication 

Any form of live interaction between 
faculty members and students via audio, 
video, or text (chat). 

Synchronous communication via text of chat function either 
within the LMS or other systems aids students m mutually 
.onstructing knowledge, negotiation, support, group facilitation, 
ir.d group processing of common course tasks (Maushak & Ou. 
2007) 

Placeholder for Faculty Use 
Examples 

Using Google Hangouts 
for Presentations 

Live video or audio communication via any usable system can 
rsult in improved faculty-student connections, reduced 
rommunication barriers possible in asynchronous disucssion. 
ind faster corrections of student work (Huang & Hsiao. 2012). 

Establish guidelines and cxpccations about what form of 
vvnehronous communication will be used in the course, provide 
tommumcation and behavior expectations, create a schedule that 
;s convicncnt for the class or targeted learning group, and 
ichcdulc low -stakes introductions to the technology being used 
Huang & Hsiao. 2012) 

Using Skvoe 

Student eMails 

Direct electronic communication to 
individual or groups of students 

rhe email students function from within the LMS provides a 
■ccord of the email directly to the faculty mailbox. This includes 
rmails to individuals, groups, or all users. 

Placeholder for Faculty Use 
Examples 

Sending Student eMail 

Assessment Feedback 

Instructor comment about student 
performance and work on assessment 
devices. This might include assignments, 
tests, quizzes, projects, or ocher graded 
activities in the course. 

Establish a time frame that inform students about when to expect 
feedback on assessment activities in the course. 

Placeholder for Faculty Use 
Examples 

Providing Assignment 
Feedback 

'eedback on Other 'ools 
in course 

Automatic ~est Feedback 



dean course discussions with low stakes introductory or 
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Student-Student Interactions 


Interaction Type 

Explanation or Definition 

Recommendations for Use 

Faculty Use Examples 

Setup Instructions 

Discussion Groups 

Asynchronous, student-student, discussion 
forums for specific sets of students usually 
assigned by the faculty member of the 
course 

Begin course discussions with low stakes introductory or 
icebreaking questions to promote familiarity amongst students 
which will support more robust and active discussions 
(Cheung ct «!.. 2008) 

Placeholder for Faculty Use 
Examples 

Creatine a Discussion Forum 

Ask or assign topics that challenge students to think and 
espond thoughtfully. Topics should not provide a clear either 
or response unless appropriate to the course topic. 

Creatine a Discussion Thread 

Provide students directions and expectations about how to 
answer and participate in discussions. This might include 
ns [ruction about asking questions for clarity and 
understanding (Cheung ct al.. 2008) 

^eolvine to a Discussion 

Establish deadlines and expectations about participation to 
foster responsible and ongoing discussions (Cheung ct al.. 
2008) 

Gradine a Discussion 

Learning Groups or 
Communities 

A subset of the full class organized around 
topics, assignments, and courscwotk. 

lave the team create a team charter and assignment plan to 
:lanfy their roles, strengths, weaknesses, and communication 
references. (Hunsakcr. Pavett. & Hunsakcr. 2011). 

Placeholder for Faculty Use 
Examples 

Creatine learnine Grouos 

Provide students support for planning assignments including 
feedback on the charter, assignment planning, and readiness 
feedback for executing the assignment (Gomez. Dezhi Wu. & 
Passcrini. 2009) 

Have students summarize communication amongst the team to 
ud both in understanding and accountability (Gomez et al.. 
2009) 

Creatine Sets of Learrire Grouos 


Include a peer evaluation (Gomez ct al.. 2009) 

Course Blogs 

Blogs. Weblogs, or Journals arc short-form 
writing areas used to capture student 
thoughts, ideas, and reflections about 
course topics. 

Create a blog for the class cither as a general blog or blog 
iround a particular topic or activity. 

Placeholder for Faculty Use 
Examples 

Creatine a Bloe 

Consider using a class blog where individuals can submit 
costings to a single blog. This promotes peer-review, 
rommentary. and dialogue amongst students (Smith. 2008) 

Creatine and Editine Bloe Entries 


Provide students instructions and expectations about 
;ontnbution including word count, format, and style. 

Commertine or Bloes 
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Icursc Dcsigr “omc 


Learning Activities 


Interaction type 


Kx plana t ion or Definition 


Recommendations for Use 


Faculty Use Examples 


Setup Instructions 


include a low-stakes or low point value quiz few each topic 
nodule of the course (Roediger 111 ct al.. 2011). 


Use a small number of questions (10) to ensure student 
»mplction of the assessment (Johnson & Kiviniemi. 2009) 


Learning Cheek Quizzes 


Quiz assessments that arc scheduled at the end 
af sets of content or course materials that 
provide students an opportunity to test their 
understanding and knowledge of the topic 
(Johnson & Kiviniemi. 2009; Roediger III. 
Agarwal . McDaniel. & McDermott. 20 1 1 ) 


Consider placing the quizzes in the module after the 
trading and ahead of any lecture materials to ensure 
.tudents have the baseline of knowledge needed for the 
ccturc (Tao. Fore. & Forbes. 2011) 


Placeholder for Faculty Use 
Examples 


Creating a Test 


In constructing the quiz include feedback for students: 
rongratulalions for correct answers and additional 
njidar.ee for incorrect answers that send students back to 
he test (Johnson & Kiviniemi. 2009). 


Use multiple choice or multiple answer questions along 
with repeated attempts to help student achieve mastery of 
he materials (Roediger 111 ct al.. 2011) 


Urcatc a blog for the class either as a general blog or blog 
iround a particular topic or activity. 


Creating a 8log 


Course Blogs 


Blogs. Weblogs, or Journals are short-form 
writing areas used to capture student thoughts 
ideas, and reflections about course topics. 


Consider using a class blog where individuals can submit 
jostmgs to a single blog. This promotes peer-review, 
rommentary. and dialogue amongst students (Smith. 2008) 


Creating and Editing Blog Entries 


Provide students instructions and expectations about 
jontribuUon including word count, format, and style. 


Placeholder for Faculty Use 
Examples 


Commenting on Blogs 


Lneourage student commenting on individual posts by 
jsir.g the comment feature and posing questions, additional 

h oughts, or additional resources. 


Feedback on Other Tools in 
course 


Suggest a Charge 
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Appendix G: Change Matrix for Instructional Delivery Framework Revisions 


Framework 

Item 

Original 

Changed 

Rationale 

Research 

Question 



Version 1 to Version 2 


Course Design 


Add Departmental and 
Institutional specifics to 
Course Design page. This 
includes both instructions 
and links to college web 
documents. 

Improved 

pedagogical quality 
by linking 
discipline specific 
and institutional 
policies about 
course instruction 
and student 
outcomes. Quality 
in pedagogy needs 
to account for 
institutional 
differences 

Appeal 

Faculty-Student 

Interactions 


Added Synchronous 
Electronic 

Communication element 

Suggestion from 
faculty that was 
vetted by research 
from Graham and 
Jones (2011) and 
Singleton and 
Session (2011) that 
faculty and 
students desire live 
or synchronous 
communication in 
an online course. 

Appeal 

Course 

Technology 

item 

Course 
Technology 
was a 
separate 
item on the 
Course 
Design 
page 

Included course 
technology instructions 
into the Course 
Instructions section 

Streamline 
document for easier 
use. Students 
require clear and 
concise instructions 
in order to meet the 
learning demands 
of online courses 
(S. A. Lei & Gupta, 
2010) 

Efficiency 

Course 

Materials 

Multiple 
similar 
forms of 
materials 
detailed out 

Consolidated items that 
had similar items with 
the same 

recommendations into 
one item with a 

Reduce the 
complexity of the 
instrument while 
maintaining the 
comprehensives. 

Efficiency 





229 




broadened description 

Improve faculty 
ease of use. Quality 
systems should be 
easy and not 
onerous to use to 
encourage faculty 
adoption (Jordens 
& Zepke, 2009; 
Shulman, 2007). 


Section Headers 

No color 
coding 

Color coding 

Enhance ease of 
use and 

organization of the 
instrument to assist 
faculty with use. 
Quality systems 
should be easy and 
not onerous to use 
to encourage 
faculty adoption 
(Jordens & Zepke, 
2009; Shulman, 
2007). 

Efficiency 

Format 

Paper- 
based only 

Added Web-based format 
as an option 

Enhance the 
flexibility and use 
of the format. 
Quality systems 
should be easy and 
not onerous to use 
to encourage 
faculty adoption 
(Jordens & Zepke, 
2009; Shulman, 
2007). 

Efficiency 

Version 2 to Version 3 


Format 

Paper and 

Web 

Formats 

Web Only 

Allows for Faculty 
Use and Setup 
Instructions to be 
included, easier 
linking between 
top page and 
different course 
components. Web- 
based tools and 
processes will 
better equip 
institutions to 

Appeal 
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monitor and 
deliver quality 
online instruction 
(Postek et al., 
2010; P. S. Smith, 
2011). 


General 

Framework 


Framework Instructions 

Provides faculty 
with expectations 
and instructions 
for using the 
framework. A 
lack of familiarity 
amongst faculty 
with the necessary 
computing 
technology, poor 
past experiences 
with teaching in 
the modality, and 
anxiety about 
using computers 
can serve as 
barriers to quality 
distance education 
(Singleton & 
Session, 2011; 
Tabata & 
Johnsrud, 2008) 

Appeal 

Interaction 

Sections 


Faculty Use Examples 

Provide users of 
the framework 
examples from 
their peers about 
how interactions 
items are used in 
their courses. 
Quality online 
teaching is more 
likely to occur 
when faculty 
members are 
supported with the 
proper instruction 
and support 
processes (Al- 
Salman, 2011; Lee 
et al., 2010). 

Effectiveness 
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Interaction 

Sections 


Setup Instructions 

Provided links to 
existing tutorials, 
instructions, and 
procedures for 
how to implement 
the item in the 
course. Quality 
online teaching is 
more likely to 
occur when faculty 
members are 
supported with the 
proper instruction 
and support 
processes (Al- 
Salman, 2011; Lee 
et al., 2010). 

Effectiveness 

General 

Framework 


Added Submit Changes 
protocol 

Enabled the 
framework to grow 
with faculty input 
and expertise. 
Faculty are best 
equipped to 
determine how and 
what should be 
used to meet 
quality objectives 
in distance 
education 
(Mancuso, 2009; 
Kirkwood & Price, 
2008) 

Effectiveness 
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Appendix H: Sample Case for Research 


- Sample Case - 

Critical Thinking 
THNK101 


Fall/Spring/Summer I/Summer II 20xx 
Course Perquisites: None 


Class Location 

Online 

Course Instructor 

= 

Instructor’s Phone 
Tnstrnrtor’s eMail 

Instructor’s Availability 

— 


Course Description 

This course provides a foundational background in critical thinking demanded by other college courses, post- 
graduation employers, and society in general. In this course you will build your ability to observe, classify, analyze, 
and synthesize information and situations in order to make meaningful decisions. Students should expect to employ 
reasoning skills, find alternatives to presented issues, consider the consequences of choices, and make decisions with 
a fully developed awareness of the issues presented in class. 

Learning Objectives 

Given the materials presented in class and through the course textbook, students will be able to: 

• Define the skills necessary to perform critical thinking in everyday situations 

• Provide examples from contemporary sources of critical and non-critical thinking 

• Respond to presented situations in a manner consistent with critical thinking 

• Explain the reasoning process used in presented situations 

• Form well-reasoned decisions and arguments based on presented situations 

• Appraise the decisions presented in class using their knowledge of critical thinking 

• Defend decisions using their knowledge of critical thinking 

• Summarize their personal knowledge and application of critical thinking for subsequent studies, post- 
graduation employment, and personal life. 


Week 

Topic 

Course 

Materials 

Faculty-Student 

Interactions 

Student- 

Student 

Interactions 

Learning 

Activities 

Student 

Assessment 

1 







2 







3 







4 







5 







6 







7 







8 







9 







10 







11 







12 







13 







14 







15 











